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Abstract 

This  dissertation  investigates  stability  conditions  and  approximate  steady-state 
performance  measures  for  unreliable,  single-server  retrial  queues  operating  in  a  ran¬ 
domly  evolving  environment.  In  such  systems,  arriving  customers  that  find  the  server 
busy  or  failed  join  a  retrial  queue  from  which  they  attempt  to  regain  access  to  the 
server  at  random  intervals.  Such  models  are  useful  for  the  performance  evaluation 
of  communications  and  computer  networks  which  are  characterized  by  time-varying 
arrival,  service  and  failure  rates.  To  model  this  time- varying  behavior,  we  study  sys¬ 
tems  whose  parameters  are  modulated  by  a  finite  Markov  process.  Two  distinct  cases 
are  analyzed.  The  first  considers  systems  with  Markov-modulated  arrival,  service, 
retrial,  failure  and  repair  rates  assuming  all  interevent  and  service  times  are  exponen¬ 
tially  distributed.  The  joint  process  of  the  orbit  size,  environment  state,  and  server 
status  is  shown  to  be  a  tri-layered,  level-dependent  quasi-birth-and-death  (LDQBD) 
process,  and  we  provide  a  necessary  and  sufficient  condition  for  the  positive  recur¬ 
rence  of  LDQBDs  using  classical  techniques.  Moreover,  we  apply  efficient  numerical 
algorithms,  designed  to  exploit  the  matrix-geometric  structure  of  the  model,  to  com¬ 
pute  the  approximate  steady-state  orbit  size  distribution  and  mean  congestion  and 
delay  measures.  The  second  case  assumes  that  customers  bring  generally  distributed 
service  requirements  while  all  other  processes  are  identical  to  the  first  case.  We  show 
that  the  joint  process  of  orbit  size,  environment  state  and  server  status  is  a  level- 
dependent,  M/G/ 1-type  stochastic  process.  By  employing  regenerative  theory,  and 
exploiting  the  M/G/ 1-type  structure,  we  derive  a  necessary  and  sufficient  condition 
for  stability  of  the  system.  Finally,  for  the  exponential  model,  we  illustrate  how 
the  main  results  may  be  used  to  simultaneously  select  arrival  and  service  rates  that 
minimize  the  mean  time  customers  spend  in  orbit,  subject  to  bound  and  stability 
constraints. 
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UNRELIABLE  RETRIAL  QUEUES  IN  A 
RANDOM  ENVIRONMENT 


1.  Introduction 

Since  the  introduction  of  the  first  digital  computers  some  70  years  ago,  networks  of 
information  systems  have  become  an  integral  part  of  the  world’s  financial,  industrial, 
educational,  and  governmental  institutions.  We  have  come  to  depend  almost  solely 
on  these  information  systems  and  the  networks  upon  which  they  reside  for  managing 
our  financial  systems,  utility  infrastructures,  and  countless  other  functions  essential 
to  modern  living.  The  negative  side  of  this  transformation  is  that  these  systems 
are  subject  to  the  unquestionably  hit-or-miss  task  of  relaying  data  through  Internet 
pathways  that  are  scarcely  controlled.  Even  the  internal  networks  of  large  and  small 
organizations  are  not  immune  from  the  often  critical  risk  of  losing  the  ability  to 
exchange  information  in  an  accurate  and  timely  fashion.  For  this  reason,  it  has 
become  necessary  to  understand  the  factors  that  influence  the  degradation,  and 
ultimately  the  failure,  of  information  systems. 

1.1  Background  and  Motivation 

Maintaining  the  integrity  of  information  networks  has  become  a  universal  con¬ 
cern  for  information  technology  managers  from  all  areas  of  society.  Corporations 
and  smaller  businesses  that  rely  on  the  internet  for  sales,  marketing,  or  other  direct 
support  must  not  only  be  able  to  communicate  with  their  clients  and  suppliers,  but 
they  must  do  so  in  a  prompt  manner ,  all  the  while  sending  and  receiving  large  quan¬ 
tities  of  information  over  global  distances.  Governmental  organizations  find  common 
ground  with  the  private  sector  in  their  desire  to  maintain  and  improve  information 
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flow.  For  example,  timely  communication  is  an  essential  staple  in  military  applica¬ 
tions.  Battle  effectiveness  is  no  longer  measured  by  sheer  volumes  of  materiel  and 
manpower,  but  rather  by  precision  strikes,  which  themselves  are  enabled  by  correct 
intelligence  and  the  effective  coordination  of  battlefield  commanders  distributed  over 
wide  geographic  areas.  The  information  systems  network  has  become,  in  fact,  the 
single  most  important  center  of  gravity  of  opposing  military  forces,  and  not  just  in 
conflict,  but  particularly  for  private  entities  and  even  nations  in  conducting  their 
day-to-day  business. 

Since  computer  networks  are  subject  to  seemingly-random  phenomena  over 
very  small  time  scales,  they  are  difficult  to  observe  directly.  It  is  therefore  advanta¬ 
geous  to  employ  mathematical  models  as  proxies  for  empirical  observations  or  direct 
experimentation.  One  such  model  that  is  frequently  used  by  analysts  in  the  com¬ 
puter  and  telecommunications  industries  is  the  queueing  system.  A  queue  consists 
of  an  input  process  for  arrivals,  a  waiting  room  (otherwise  known  as  a  buffer ),  and 
servers  that  process  the  arrivals.  The  input  is  itself  a  random  process  that  describes 
the  timing  of  arrivals  to  the  queue,  and  is  also  characterized  by  the  behavior  of  these 
arrivals;  for  instance,  these  may  arrive  individually  or  in  groups  (batches),  or  they 
may  choose  to  renege  and  abandon  the  queue  entirely.  The  service  mechanism  of  a 
queueing  system  is  likewise  determined  by  a  random  process,  but  one  that  describes 
the  duration  of  service  episodes.  It  is  furthermore  regulated  by  a  queueing  discipline 
which  dictates  the  order  in  which  arriving  entities  are  processed.  Either  or  both  of 
the  input  and  service  processes  may  be  deterministic,  as  in  an  automated  production 
line,  or  probabilistic,  as  in  a  traffic  flow  model.  In  the  classical  queueing  models, 
servers  are  usually  assumed  to  be  reliable.  Therefore,  entities  that  leave  the  system 
do  so  only  because  they  have  received  service,  and  then  they  depart  permanently. 
However,  it  is  apparent  that  the  assumption  of  failure-free  operation  is  not  a  re¬ 
alistic  one,  and  so  the  classical  system  fails  to  incorporate  an  essential  stochastic 
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characteristic:  server  failure.  Understanding  systems  in  which  such  failures  occur  is 
particularly  important  to  those  who  analyze  communication  networks. 

It  may  be  unrealistic  to  expect  entities  to  be  lost  by  a  system  even  if  the  server 
is  busy  or  has  failed.  Current  data-link  protocols,  for  instance,  usually  employ 
a  handshake  procedure  to  verify  that  a  successful  communication  has  been  made, 
during  which  time  the  sent  data  is  stored  in  a  buffer  for  later  re-transmission  if 
necessary.  Such  a  protocol  is  described  in  [110]  and  operates  in  the  following  manner. 
The  sending  device  starts  off  the  process  by  transmitting  an  information  packet  to 
the  receiving  device.  It  is  important  to  note  here  that  it  keeps  the  packet  until  it  has 
received  verification  from  the  receiver  that  it  has  arrived  uncorrupted,  a  task  which 
is  accomplished  via  the  checksum  method.1  If  the  receiver  determines  that  it  has 
received  the  packet  correctly,  then  it  sends  an  acknowledgment  (ACK)  back  to  the 
sending  device,  together  with  bits  that  will  allow  the  sender  to  verify  the  integrity 
of  the  ACK.  If,  in  the  course  of  this  exchange,  the  packet  or  the  ACK  becomes 
corrupted,  then  the  sender  will  determine  that  it  did  not  receive  a  valid  ACK.  It  will 
then  wait  for  a  prescribed  timeout  period  in  order  to  determine  whether  or  not  a 
transmission  error  has  occurred  and  resend  the  packet.  In  the  parlance  of  queueing 
theory,  such  a  mechanism  in  which  ejected  (or  rejected)  customers  return  at  random 
intervals  until  they  receive  service  is  called  a  retrial  queue.  Retrial  queues  have 
application  in  a  wide  variety  of  fields,  and,  as  this  example  clearly  shows,  they  are 
particularly  useful  in  describing  communication  systems. 

A  retrial  queue  is  similar  to  an  ordinary  queueing  system  in  that  there  is 
an  arrival  process  and  one  or  more  servers.  The  fundamental  differences  are  that 
(i)  entities  who  enter  during  a  down  or  busy  period  of  the  server  or  servers  may 
reattempt  service  at  some  random  time  in  the  future,  and  (ii)  a  waiting  room,  which 

1The  checksum  method  is  used  for  the  detection  of  errors  in  transmitted  messages.  A  compu¬ 
tation  is  performed  on  the  essential  bits  of  a  message,  stored,  and  then  transmitted  along  with  the 
message.  The  same  computation  is  performed  on  the  receiving  end,  and  a  comparison  is  made  to 
the  transmitted  checksum  information. 
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is  known  as  a  primary  queue  in  the  context  of  retrial  queues,  is  not  mandatory. 
In  place  of  the  ordinary  waiting  room  is  a  buffer  called  an  orbit  to  which  entities 
proceed  after  an  unsuccessful  attempt  at  service,  and  from  which  they  retry  service 
according  to  a  given  probabilistic  or  deterministic  policy.  Note  that  the  orbit  may 
or  may  not  be  capacitated  depending  upon  the  application.  More  likely  than  not, 
orbits  that  simply  represent  a  pool  of  returning  customers  rather  than  an  actual 
physical  waiting  area  will  have  an  unbounded  capacity.  A  typical  example  of  an 
infinite-capacity  retrial  queue  orbit  would  be  the  pool  of  customers  dialing  into  a 
call  center  who,  having  abandoned  their  service  request  after  waiting  (i.e.  left  the 
primary  queue),  try  again  at  a  later  time  (i.e.  join  the  retrial  orbit). 

A  key  distinguishing  feature  of  retrial  queues  is  the  way  in  which  retrials  are 
conducted.  Customers  may  reattempt  service  independently  of  each  other  at  random 
or  fixed  intervals  of  time,  or  they  may  follow  more  structured  policies.  For  instance, 
one  might  impose  a  queueing  discipline  such  as  ‘ first-in-first-out ’  (FIFO)  or  ‘last- 
in-first-out’  (LIFO)  upon  the  entities  in  the  orbit.  Yet  another  policy  is  to  classify 
arrivals  to  the  retrial  queue  and  establish  a  prioritization  scheme  based  upon  class 
membership.  Some  systems  may  also  incorporate  impatient  customers  who  will  retry 
a  finite  number  of  times  or  for  a  fixed  time  interval2  (which  could  have  zero  duration) 
and  then  leave  the  system  permanently.  Any  combination  of  these  priorities  may 
be  imposed  as  well,  but  the  analyst  must  keep  in  mind  the  analytical  complexity 
that  these  features  may  introduce  into  the  system  description  and  the  numerical 
computation  of  the  system  measures. 

A  classic  example  of  a  retrial  queue  application  may  be  found  in  the  description 
of  caller  behavior  in  cellular  telephone  networks.  In  the  basic  model,  a  cellular  service 
area  is  partitioned  into  subareas  known  as  cells,  each  of  which  contains  a  base  station 

2The  time  interval  may  vary  randomly  for  each  customer  or  be  a  deterministic  value  that  is 
assigned  by  class  or  some  other  criteria. 
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that  in  turn  patches  each  call  that  it  handles  into  the  national  phone  network.3 
Calls  that  originate  in  one  cell  but  move  into  an  adjoining  cell  are  assigned  a  certain 
frequency  in  the  new  cell  should  one  exist.  If  one  does  not  exist,  then  the  call  is 
dropped.  The  underlying  assumption  of  the  retrial  model  is  that  the  caller  will  try 
again  and  then  leave  the  system  when  the  conversation  is  terminated.  Extensions  of 
this  model  may,  for  example,  include  impatient  or  prioritized  callers  as  well  (consider 
911).  Regardless,  it  is  not  unreasonable  to  assume  that  persons  who  initiate  calls  do 
so  independently  of  each  other. 

The  net  effect  of  the  partitioning  of  a  service  area  in  a  cellular  network  is  an 
increase  in  system  call  capacity  that  ultimately  derives  from  the  reuse  of  frequencies 
amongst  the  various  cells.  Borst,  et  al.  [23]  describe  a  modified  version  of  the  orig¬ 
inal  cellular  network  architechture  called  a  layered  network  that  further  augments 
network  capacity.  In  this  arrangement,  cells  are  themselves  partitioned  into  what 
are  termed  microcells.  These  sub-partitions  are  optimally  used  to  cover  small  areas 
that  experience  a  high  volume  of  calls,  which,  in  the  jargon  of  the  cellular  communi¬ 
cations  industry,  are  known  as  hotspots.  Calls  that  are  blocked  at  the  microcell  level 
are  sent  to  overflow  buffers  at  the  macrocell  level.  Here  the  call  waits  until  it  is  either 
assigned  a  channel  by  the  macrocell  or  a  channel  opens  up  in  the  original  microcell 
(repacking) .  Retrials  correspond  here  to  the  repacking  process,  which  occurs  accord¬ 
ing  to  constant  exponential  retrial  rates  that  depend  upon  the  overflow  buffer.  Such 
a  scheme  tremendously  increases  the  capacity  of  each  cell  since  (i)  channels  may 
be  reused  more  frequently,  and  in  places  where  they  are  most  needed,  and  (ii)  the 
repacking  scheme  frees  up  additional  channels  in  the  macrocell  for  use  elsewhere. 

Retrial  queues  are  likewise  prevalent  in  the  evaluation  and  design  of  computer 
networks  as  they  are  in  telecommunications.  Libman  and  Orda  [72]  describe  an  op¬ 
timal  scheme  for  connections  to  internet  websites  from  client  computers.  One  of  the 

3In  the  telecommunications  jargon,  this  is  often  referred  to  as  ‘POTS’,  which  is  an  acronym  for 
‘plain  old  telephone  system’. 
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many  aggravating  issues  for  users  of  the  internet  is  the  problem  of  excessively  long 
wait  times  when  attempting  connections  to  slow  or  inoperative  web  servers.  This 
wait  duration,  known  formally  as  the  timeout  duration,  is  defined  as  the  maximum 
time  that  a  server  allows  between  the  receipt  of  a  connection  request  and  the  acknowl¬ 
edgement  by  the  server  of  failure  or  success.  Current  protocols  set  a  conservatively 
large  timeout  in  order  to  guarantee  a  high  probability  of  receipt  of  acknowledgement. 
This  policy  works  reasonably  well  in  systems  with  a  low  blocking  probability,  but 
otherwise  creates  unnecessarily  lengthy  waiting  periods  before  a  ‘failure-to-connect’ 
is  issued.  The  optimal  policy  turns  out  to  be  one  in  which  the  server  retries  the  con¬ 
nection  one  or  more  times  before  embarking  on  the  timeout  period.  This  reduces  the 
expected  time  to  success  (since  retrials  may  take  different  connection  paths)  while 
assigning  little  additional  work  to  the  server.  The  effectiveness  of  such  a  policy  is 
certainly  familiar  to  anyone  who  has  attempted  to  connect  to  a  slow  website. 

Yet  another  application  of  the  retrial  queueing  model  to  computer  networking, 
and  particularly  to  wireless  networking,  is  described  in  [110].  The  problem  in  this 
case  is  to  determine  a  protocol  that  will  facilitate  communication  between  clients 
and  servers  in  a  multiple-  accessnetwork,  which  denotes  a  system  of  networked  clients 
that  use  a  single  communication  channel.4  Since  clear  transmission  frequencies  are 
hard  to  come  by,  it  is  often  necessary  to  resort  to  the  use  of  a  single  channel  for 
transmitting  data  packets.  As  a  result,  there  must  be  some  form  of  protocol  in  place 
that  permits  the  network  to  function  despite  the  inevitability  of  cross-interference 
that  occurs  when  two  or  more  clients  attempt  to  transmit  through  the  same  channel 
simultaneously.  There  are  several  such  protocols  that  are  being,  or  have  been  used, 
during  the  past  40  years,  the  first  and  most  straightforward  of  which  is  known  as  the 
ALOHA  protocol,  which  was  developed  at  the  University  of  Hawaii  during  the  early 
1970s.  When  two  computers  in  a  network  utilizing  this  protocol  transmit  a  packet 

4The  ‘channel’,  as  it  is  known  in  computer  networking  terminology,  is  synonymous  with  the 
queue  in  the  corresponding  stochastic  model.  In  the  context  of  wireless  networking,  the  channel 
corresponds  to  the  radio  frequency  used  by  the  network  for  communication  amongst  clients. 
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at  the  same  time,  the  subsequent  interference  is  detected  by  both  computers,  after 
which  each  independently  waits  a  random  time  before  re-transmitting  their  packets. 
The  inefficiency  of  this  method  results  from  the  fact  that  the  whole  packet  is  always 
transmitted  before  detection  may  take  place.  The  Ethernet  protocol,  on  the  other 
hand,  uses  what  is  called  the  ‘Carrier  sense  multiple  access  with  collision  detection’, 
or  CSMA-CD,  to  sense  when  packets  are  in  the  transmission  channel.  This  enables 
the  network  to  quickly  detect  the  occurrence  of  simultaneous  transmissions  and  then 
terminate  the  transmissions  before  a  complete  packet  has  been  sent. 

No  system  resides  or  operates  in  a  vacuum,  and  information  systems  are  like¬ 
wise  influenced  by  an  environment,  which  may  be  described  as  being  either  physical 
(hardware-related)  or  virtual  (network-related).  Of  a  certainty,  the  most  critical 
and/or  fragile  systems  must  physically  reside  in  controlled  environments  in  order  to 
protect  the  servers  and  network  hardware  from  the  untempered  elements.  That  we 
take  these  precautions  stems  from  our  knowledge  that  the  net  effect  of  various  factors 
such  as  temperature,  humidity,  and  shock  results  in  a  degradation  of  the  hardware 
over  time,  or  may  even  cause  sudden  or  catastrophic  failure  to  occur.  Further¬ 
more,  certain  combinations  of  environmental  factors  inflict  such  damage  at  greater 
rates  than  others,  and  so  we  bank  our  investment  in  hardware  on  the  assumption 
that  controlling  the  environment,  in  other  words,  regulating  the  stochastic  factors  of 
temperature,  humidity,  etc.,  will  mitigate  the  risk  of  total  system  failure. 

In  the  context  of  networking,  it  is  just  as  important  to  understand  the  user 
environment,  for  this  determines  the  patterns  of  data  flow,  and  thus  the  frequency 
of  occurrence  of  certain  events  such  as  packet  collisions  across  a  channel.  Larger 
networks  tend  to  experience  such  phenomena  in  direct  proportion  to  the  number  of 
users  due  to  the  fact  that  storage  and  link  capacities  are  finite,  and  expensive  at 
that.  Moreover,  variations  in  the  number  of  users  at  different  times  will  likely  affect 
the  speed  at  which  information  is  disseminated  within,  or  transferred  out  of,  a  given 
network.  In  developing  a  stochastic  model  of  such  a  system,  one  may  opt  to  ignore 
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the  minutiae  of  these  variations  by  assigning  average  rates  of  data  packet  arrivals 
and  transmission;  indeed,  this  may  be  sufficient  if  the  network  is  small,  sees  little 
variation,  or  has  a  large  capacity  with  respect  to  the  number  of  authorized  users. 
In  other  scenarios,  however,  the  greater  sensitivity  of  system  parameters  to  such 
variations  makes  it  necessary  to  include  them  in  the  model.  An  example  of  such 
a  system  would  be  a  group  of  web  servers  that  serve  a  nationwide  customer  base. 
Such  systems  often  operate  at  or  near  capacity  during  peak  hours,  and  the  resulting 
strain  on  the  system  exhibits  itself  in  dramatically  decreased  data-transfer  rates.  At 
other  times,  the  transfer  rates  will  be  at  the  maximum  that  the  physical  hardware 
can  support.  Modeling  such  networks  with  an  adequate  degree  of  fidelity  may  thus 
entail  the  adjustment  of  service  rates  based  on  the  time-of-day. 

One  alternative  for  incorporating  the  effects  of  external  environmental  factors 
into  a  stochastic  model  is  by  using  a  random  environment.  The  random  environment 
process  is  itself  a  stochastic  process  whose  state  is  usually  assumed  to  be  independent 
of  the  state  of  the  process  that  it  influences.  It  may  take  a  number  of  varied  forms 
such  as  a  discrete-  or  continuous-time  Markov  chain,  a  random  walk,  a  semi-Markov 
process,  or  Brownian  motion,  to  name  a  few.  These  mathematical  objects  are  also 
called  modulating  processes.  Consequently,  if  the  random  environment  is  Markovian, 
the  primary  stochastic  process  to  which  it  is  attached  is  said  to  be  Markov-modulated. 
One  might  also  see  the  key  phrases  ‘varying  randomly’,  ‘state-dependent’  (if  one  is 
referring  to  the  state  of  an  external  modulating  process,  not  an  internal  system 
state),  or  ‘in  a  Markovian  environment’.  There  are  other  synonyms  for  the  same 
exogenous  environmental  process. 

A  random  environment  can  be  used  to  modulate  any  of  the  system  parameters, 
which  in  the  context  of  a  retrial  queue  queue  might  include  the  arrival,  service,  or 
retrial  rates.  If  another  process  -  such  as  one  that  provides  the  timing  of  server 
failures  -  feeds  into  the  same  system,  then  the  random  environment  may  be  used 
to  modulate  its  parameters  as  well.  As  may  be  expected,  the  dependence  of  system 


parameters  upon  the  random  environment  is  implemented  via  the  use  of  functional 
relationships  with  time  as  the  dependent  variable.  The  most  straightforward  way 
to  define  such  a  function  is  to  simply  assign  rates  deterministically  according  to 
the  state  of  the  random  environment.  More  uncertainty  may  be  introduced  into  the 
stochastic  model  by  constructing  a  process  that  assigns  the  rates  randomly  according 
to  a  stream  taken  from  a  probability  distribution.  Regardless,  care  must  be  taken  to 
distinguish  the  external  random  process  from  the  internal  version  in  which  the  rates 
depend  upon  the  states  of  the  primary  system.  This  latter  object  is  often  called 
a  state- dependent  model,  although  it  appears  in  the  literature  occasionally  in  the 
random  environment  context. 

It  is  easy  to  see  the  relevance  of  this  combination  of  the  retrial  queueing  model 
and  an  external  random  environment.  For  a  physical  environment,  one  may  use  a 
degradation  model  in  which  the  effects  of  heat,  humidity,  radiation,  and  other  effects 
produce  cumulative  damage,  to  include  electronic  hardware  and  other,  more  solid- 
state  items.  For  the  ethernet  model,  one  may  have  incidents  of  network  congestion 
occur  on  the  heels  of  arrivals  in,  say,  an  external  Poisson  process,  or  the  transmission 
rate  may  vary  according  to  the  state  of  a  semi-Markov  process.  If  one  is  confident 
that  the  random  process  modulates  rates  or  otherwise  behaves  in  a  manner  similar 
to  that  of  the  actual  system,  then  one  might  expect  that  the  state  measures  derived 
from  such  a  model  will  accurately  predict  how  the  system  will  behave  under  the 
assumed  conditions. 

1.2  Research  Objectives 

Owing  to  the  utility  and  interesting  mathematical  properties  of  retrial  queueing 
models,  a  vast  literature  on  the  subject  has  emerged  over  the  past  several  decades. 
However,  relatively  few  researchers  have  touched  upon  the  subject  of  unreliable 
servers  in  the  retrial  context,  and  even  fewer  have  considered  the  impact  of  a  ran¬ 
domly  evolving  operating  environment  on  the  performance  measures  of  such  systems. 
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A  primary  goal  of  this  research  is  to  advance  the  theory  of  retrial  queues  by  consid¬ 
ering  systems  whose  operating  parameters  (e.g.,  arrival,  service,  retrial,  failure  and 
repair  rates)  are  modulated  by  a  time-varying  environment.  The  main  objectives  of 
this  dissertation  can,  therefore,  be  summarized  as  follows: 

1.  To  develop  basic  results,  such  as  stability  conditions  and  performance  mea¬ 
sures,  for  unreliable,  single-server  retrial  queues  that  operate  in  a  random 
environment  assuming  customers  bring  an  exponentially  distributed  service 
requirement  to  the  system; 

2.  To  extend  the  results  obtained  in  Objective  (1)  to  consider  systems  in  which 
customers  bring  generally  distributed  service  requirements  to  the  system; 

3.  To  illustrate  how  the  main  results  can  be  used  to  improve  the  performance  of 
such  systems  by  optimally  selecting  operating  parameters  that  minimize  the 
steady-state  mean  time  spent  in  the  retrial  orbit. 

1.3  Dissertation  Outline 

In  the  next  chapter,  we  review  the  literature  pertinent  to  general  retrial  queues, 
retrial  queues  with  unreliable  servers,  and  general  (non-retrial)  queues  operating  in 
a  random  environment.  In  Chapter  3,  the  rudimentary  concepts  of  the  class  of 
quais-birth-and-death  (QBD)  processes,  as  well  as  Markov  chains  of  the  M/G/l- 
type,  are  reviewed.  These  concepts  serve  as  a  foundation  for  the  main  stability 
result  of  the  exponential  model  in  Chapter  4  which  provides  valuable  insights  into 
the  conditions  needed  for  positive  recurrence  of  generalized  lev  el- dependent  QBD 
processes.  Chapter  5  extends  the  results  of  Chapter  4  by  considering  customers  who 
bring  a  generally  distributed  service  requirement,  an  extension  that  adds  considerable 
complexity  to  the  analysis  of  the  model.  This  model  possesses  an  embedded  level- 
dependent  Markov  chain  of  M/G/l-type,  and  we  prove  conditions  for  the  stability 
of  this  complex  system.  The  sixth  chapter  illustrates  how  the  results  of  Chapter  4 
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can  be  used  to  enhance  the  performance  of  unreliable  retrial  queueing  systems  by 
optimizing  the  arrival  and  service  rates  associated  to  the  distinct  environment  states. 
The  seventh  and  final  chapter  summarizes  the  main  contributions  of  this  dissertation 
and  provides  some  directions  for  future  work  in  this  area. 
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2.  Review  of  the  Literature 

Retrial  queueing  comprises  a  significant  portion  of  the  modern  literature  on  queueing 
theory.  For  a  general  survey  of  retrial  queues  and  a  summary  of  many  results,  the 
reader  is  directed  to  the  works  of  Yang  and  Templeton  [113],  Falin  [38]  and  refer¬ 
ences  therein.  The  wide  range  of  applications  of  retrial  queues  with  unreliable  servers 
has  generated  much  interest  among  stochastic  systems  researchers.  On  a  seemingly 
different  note,  the  fundamental  relationship  between  the  reliability  of  an  item  and 
its  working  environment  has  popularized  research  into  random  environments  since 
the  late  1940s.  The  high  level  of  attention  that  each  of  these  separate  subject  areas 
has  received  has  not,  however,  produced  the  simultaneous  pairing  of  retrial  queue¬ 
ing  systems  with  unreliable  servers  whose  parameters  are  modulated  by  a  random 
environment.  In  this  chapter,  we  shall  review  the  literature  that  pertains  to  these 
unrelated  research  tracks  and  the  emerging  literature  that  begins  to  suggest  their 
combination. 

2.1  Analysis  and  Control  of  Retrial  Queues 

A  retrial  queue,  as  defined  by  Falin  in  [38] ,  is  essentially  a  queueing  system  with 
no  waiting  room  in  which  blocked  entities,  or  customers,  may  revisit  the  server  at 
some  random  time  in  the  future.  Thus,  these  secondary  (retrial)  entities,  do  not  leave 
the  system,  but  instead  proceed  to  a  buffer,  which  in  the  context  of  retrial  queues  is 
termed  an  orbit.  No  stipulation  is  placed  on  the  structure  of  the  orbit  nor  the  policies 
that  specify  how  entities  retry  the  server  or  how  they  are  released  from  the  system. 
In  particular,  the  leeway  granted  in  determining  the  nature  of  the  blocking,  whether 
it  be  a  busy  or  failed  server,  and  the  mechanism  by  which  blockings  occurs  leads  to  a 
wide  variety  in  the  open  literature  on  the  topic.  In  general,  however,  the  structure  of 
the  orbit  takes  one  of  two  forms.  The  first  is  a  queueing  structure  for  which  entities 
are  released  back  to  the  server  according  to  a  first-in-first-out  (FIFO)  discipline.  The 
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other  type  is  that  of  an  infinite-server  queue  in  which  each  retrial  entity  revisits  the 
server  after  some  randomly-distributed  duration  but  independently  of  every  other 
retrial  entity.  This  paradigm  is  appropriate  for  modeling  such  real-world  systems  as 
computer  networks  in  which  the  entities  -  data  packets  -  are  homogeneous  and  no 
ordering  protocol  is  defined. 

The  seminal  papers  in  retrial  queueing  were  published  by  Kosten  [58],  Clos 
[30],  Wilkinson  [112],  and  Cohen  [31,  32],  each  of  whom  were  working  to  solve  block¬ 
ing  issues  in  telephone  networks.  The  key  to  their  research  was  the  determination  of 
performance  measures  related  to  blocking  probabilities;  for  example,  the  probability 
of  having  n  busy  trunks  (primary  paths  in  a  telephone  network)  and  the  percentage 
of  dropped  calls.  In  a  1957  paper  by  Cohen  [32],  the  entities  are  calls  that  pos¬ 
sess  independent  and  identically  distributed  (i.i.d.)  general  inter-arrival  times  with 
exponential  holding  times  (in  service).  Calls  may  take  any  idle  trunk;  if  no  such 
trunk  exists,  then  the  call  may  either  be  discarded  or  sent  to  an  overflow  trunk  - 
which  is,  of  course,  the  retrial  orbit.  This  system  is  useful  in  modeling  a  telephone 
caller  who,  once  blocked  by  the  system  from  placing  his  call,  may  redial  at  some 
indeterminate  time  in  the  future.  As  Clos  [30]  observes,  the  preponderance  of  callers 
will  in  fact  try  to  call  again,  which  is  synonymous  with  ‘will  not  leave  the  system’. 
See  [30,  31,  32,  58,  112]  for  further  details  on  the  authors’  respective  models  and 
analytical  approaches. 

The  retrial  process  as  envisioned  by  Kosten  [58],  Clos  [30],  Wilkinson  [112]  was 
extended  to  single-server  queues  with  no  waiting  room  and  general  service  times  - 
in  other  words,  a  retrial  queue  based  on  the  M/G/l/1  model  -  by  Keilson,  Coz- 
zolino,  and  Young  [48].  Retrial  times,  as  well  as  primary  inter-arrival  times,  follow 
exponential  distributions.  The  authors  derive  the  basic  asymptotic  measures  for  the 
one-server  case  based  upon  transient  results  derived  by  Keilson  and  Kooharian  in 
[49]  for  the  standard  M/G/l  queue. 
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The  next  significant  result  was  published  in  1983  by  V.G.  Kulkarni  [60].  In  this 
article,  he  proves  a  seemingly  simple  equilibrium  result  A R  =  A SRS1  which  basically 
states  that,  in  the  long  run,  the  mean  number  of  total  unsuccessful  attempts  by 
incoming  customers  equals  the  number  of  such  attempts  during  service  periods.  He 
then  uses  this  fact  to  obtain  the  mean  number  of  differentiated  customers  in  each  of 
two  classes  in  an  M/G/l  system  with  different  exponential  retrial  times  and  arrival 
rates.  Choi  and  Park  [29]  follow  on  the  heels  of  this  result  in  their  1990  paper.  Their 
retrial  model  consists  of  a  primary  queue  and  orbit,  each  of  infinite  capacity,  and  a 
Bernoulli  splitting  mechanism  that  routes  a  proportion  of  blocked  customers  to  the 
orbit.  Falin  and  Artalejo  [39]  expand  the  analysis  of  finite  source  retrial  queues1  to 
include  the  waiting  time  process  and  an  excursion  into  transient  measures  via  the 
busy  period.  Finally,  Kumar  and  Arivudainambi  [64]  retrial  queues  consider  the 
M/G/l  retrial  model  in  which  server  vacations  are  controlled  by  a  Bernoulli  process 
and  the  orbit  is  governed  by  a  FIFO  discipline  in  which  only  the  customer  at  the  head 
of  the  orbit  queue  is  allowed  to  access  the  server.  The  Bernoulli  vacation  process  is 
of  special  interest  as  it  relates  directly  to  the  topic  of  failures  in  retrial  queues,  which 
we  shall  introduce  next. 

2.2  Retrial  Queues  With  Unreliable  Servers 

In  the  conventional  retrial  queueing  model,  customers  proceed  to  the  orbit  if 
they  find  all  of  the  servers  busy  upon  entering  the  system.  It  is  a  natural  extension 
to  consider  failures  of  the  server,  particularly  in  light  of  the  relevance  of  such  a 
model  to  the  reliability  study  of  real-world  systems.  Much  of  the  terminology  in  the 
literature  regarding  ordinary  queues  with  unreliable  servers  has  passed  on  to  their 
counterparts  in  retrial  queues.  In  this  case,  it  becomes  important  to  distinguish 
between  failures  that  occur  during  service  (‘active  breakdowns’)  and  those  that  occur 
while  the  server  is  idle  (‘nonactive  breakdowns’)  since  this  will  obviously  impact  the 

Hn  other  words,  there  exist  a  finite  number  of  possible  arrivals  to  the  queue. 


14 


orbit  size.  Realistically  speaking,  failures  occur  probabilistically,  with  failure  states 
represented  in  the  model  by  states  in  a  stochastic  process  (such  as  a  Markov  chain). 
This  scheme  may  be  simplified  further  via  the  use  of  alternating  states  in  which  a 
two-state  stochastic  process  governs  failure  in  an  on-off  fashion.  Accordingly,  the 
server  is  assigned  a  rate  of  repair  that  is  independent  of  the  retrial  rate  of  customers 
in  the  orbit,  while  arrival  and  service  rates  depend  upon  the  state  of  a  discrete 
modulating  process.  This  creates  an  interesting  dynamic  in  which  such  measures  as 
the  orbit  size,  customer  throughput,  and  sojourn  time,  become  the  primary  focus  of 
investigation. 

The  first  mention  of  retrial  queues  with  unreliable  servers  appeared  in  an  article 
by  Yang  and  Templeton  [113],  which  is  a  survey  of  retrial  queues  in  the  spirit  of 
Falin  [38],  but  which  additionally  associates  the  breakdown- related  topic  of  retrial 
queues  to  server  vacations.2  Nevertheless,  Aissani  [4]  is  credited  with  developing  the 
first  retrial  queueing  model3  that  included  server  breakdowns  in  his  seminal  article 
published  in  1988.  The  author  introduced  two  variations  of  the  unreliable  retrial 
model:  (1)  the  queue  whose  server  breakdowns  occurs  according  to  a  Poisson  process, 
and  (2)  the  same  queue,  but  with  failures  determined  by  a  two-state  Markov  chain. 
He  then  generates  the  performance  measures  of  interest  via  probability  generating 
functions  (p.g.f.). 

Two  years  later,  Kulkarni  and  Choi  [62]  independently  derived  results  for  an 
M/G/l/1  retrial  queue  with  exponentially-distributed  retrial  and  failure  times.  Just 
as  in  [60],  the  failure  rate  is  modulated  by  the  status  of  the  server,  where  by  ‘status’ 
the  authors  mean  ‘idle’  or  ‘busy’.  In  the  meantime,  Aissani  [4]  continued  to  build 
upon  the  results  he  had  obtained  up  to  1988.  In  the  article  that  follows,  Aissani  [5] 
re-examined  the  first  of  the  two  models  studied  previously  in  [4]  by  generalizing  the 
distributions  of  the  failure  times  and  by  specifying  batch  Poisson  arrivals.  He  demon- 

2The  distinction  here  is  that  vacations  are  server  breakdowns  that  occur  when  the  server  is  idle. 

3The  author  refers  to  retrials  here  as  repeated  calls.  Other  synonyms  for  retrial  include  repeated 
orders,  repeated  attempt,  and  returning  customer. 
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strated  that  the  ergodicity  conditions  hold  for  the  corresponding  three-dimensional 
system  evolution  process.  He  then  derived  an  expression  for  the  ^-transform  of  the 
long-run  size  of  the  secondary-source  buffer  (orbit). 

Aissani  [6]  continued  to  elaborate  upon  his  model  with  the  addition  of  redun¬ 
dant  servers  that  substitute  for  failed  primary  servers  during  a  corresponding  repair 
epoch.  This  was  followed  by  a  paper  that  he  co-authored  with  Artalejo  in  1998 
[7]  in  which  he  introduces  a  modified  version  of  the  M/G/l/1  queue  that  appeared 
in  its  original  form  in  both  [4]  and  [62].  In  that  paper,  the  system  is  subject  to 
exponentially-distributed  server  failures  in  which  distinct  failure  rates  apply  accord¬ 
ing  to  whether  the  server  is  busy  or  idle.  They  also  introduced  the  auxiliary  queueing 
system  in  which  interrupted  customers  have  the  choice  either  to  leave  the  system 
or  to  remain.  Lastly,  the  authors  introduced  the  concept  and  terminology  of  the 
fundamental  server  period ,  which  is  defined  as  the  period  of  time  between  the  start 
of  service  and  the  next  time  that  the  server  is  available  to  begin  processing  another 
customer.  Note  that  the  interval  of  time  corresponding  to  a  fundamental  server  pe¬ 
riod  does  not  necessarily  end  with  the  conclusion  of  the  current  customer’s  service  in 
an  unreliable  retrial  system.  To  be  more  specific,  if  service  is  interrupted,  then  this 
interval  of  time  ends  at  the  instant  that  the  server  becomes  operational  (i.e.  after 
repair) . 

The  years  following  1994  are  marked  by  steady  progress  in  the  development  of  a 
variety  of  queues  with  unreliable  servers  as  evidenced  by  the  numerous  contributions 
published  during  that  time  period.  Of  notable  mention  is  the  work  of  Sherman 
and  Kharoufeh  [96],  in  which  results  are  derived  for  an  M/M/1  retrial  queue  with 
unreliable  server  and  infinite-capacity  primary  queue  and  retrial  orbit.  The  authors 
derive  explicit  expressions  for  the  limiting  distributions  of  orbit  size,  queue  size,  and 
number  in  the  system  via  a  p.g.f.  approach.  Other  useful  references  from  this  fruitful 
period  of  time  can  be  found  in  [9,  11,  14,  35,  42,  52,  59,  63,  65,  70,  71,  97,  102,  111], 
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2.3  Queueing  Systems  in  a  Random  Environment 


In  the  context  of  queueing  theory,  the  term  ‘random  environment’  refers  to  a 
stochastic  process  that  controls,  or  modulates,  one  or  more  of  the  parameters  of  the 
queue.  In  other  words,  it  is  an  exogenous  process  whose  evolution  is  independent 
of  the  process  it  modulates.  If  the  external  random  process  is  Markovian,  then  the 
primary  stochastic  process  is  said  to  be  Markov-modulated,  and  it  is  this  form  of 
random  environment  that  is  most  often  seen  in  the  queueing  literature,  though  it  is 
certainly  not  the  only  one.  The  number  of  classifications  into  which  the  work  to-date 
on  stochastic  systems  operating  in  random  environments  is  vast.  Hence,  the  reader 
is  referred  to  [21,  34,  43,  55,  56,  90,  107]  for  further  reading. 

The  modulating  random  process  possesses  the  natural  interpretation  of  being 
an  external  environment  that  instigates  change  in  some  system  parameter  of  interest. 
As  an  example,  one  might  consider  the  effects  of  ambient  temperature  and  humidity 
on  the  proper  operation  of  a  circuit  board  to  be  an  interpretation  of  the  analytical 
notion  of  an  external  environment.  This  connection  makes  queueing  in  a  random 
environment  critical  to  the  analytical  modeling  of  real-world  environmental  effects, 
and  which  leads  to  the  alternate  moniker  ‘random  environment’.  In  what  follows,  we 
will  discuss  the  important  trends  and  results  over  the  past  40  years  since  the  seminal 
papers  on  random  processes  in  queueing  have  appeared.  This  will  make  apparent 
the  current  open  problems  involving  queues  in  random  environments,  and  thus,  pave 
the  way  for  the  contributions  that  will  be  made  in  this  dissertation. 

Random  environments,  as  they  appear  in  the  literature,  are  cast  into  the  form 
of  virtually  every  stochastic  process  known  using  every  possible  modulating  mecha¬ 
nism.  The  earliest  methods  for  incorporating  a  random  environment  into  a  stochastic 
model  were  either  to  use  a  random  draw  of  service  (or  arrival)  rates  discretely  (using 
Bernoulli  trials)  or  in  accordance  with  some  continuous  distribution.  Later,  finite- 
state  Markov  chains,  and  even  semi-Markov  processes,  were  employed  as  modulating 
processes.  Accordingly,  random  environments  appeared  under  a  variety  of  names, 
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such  as  ‘variable  service  and  arrivals’  and  even  as  specialized  as  ‘wear  processes’. 
Researchers  have  considered  nearly  every  possible  facet  of  these  environments,  not 
only  for  their  intrinsic  interest,  but  also  for  the  fidelity  that  these  models  afford  to 
analysts  studying  real-world  systems. 

The  seminal  articles  on  this  subject  were  published  by  four  sets  of  authors  over 
a  period  of  a  decade.  The  first  batch  that  appeared  in  1963  consisted  of  works  by 
Eisen  and  Tainiter  [37],  Avi-Itzhak  [18],  and  Avi-Itzhak  and  Naor  [19].  This  was 
followed  in  1966  by  the  work  of  Leese  and  Boyd  [69],  and,  finally,  in  1971  by  Yechiali 
and  Naor  [115],  who  independently  worked  on  the  same  problem  as  the  one  that 
appeared  in  [37].  After  the  initial  foundations  were  laid  in  the  intervening  ten- year 
period  since  1963,  researchers  began  to  experiment  with  various  combinations  of 
stochastic  processes  and  random  environments.  Stochastic  researchers  from  other 
fields  picked  up  the  topic,  and  soon  thereafter,  it  became  a  staple  in  the  stochastic 
elements  of  every  physical  science,  and  particularly  all  fields  of  engineering  and 
reliability. 

Among  the  first  articles  on  stochastic  systems  in  a  random  environment  is 
that  of  Eisen  and  Taineter  [37].  Their  model  is  a  single-server  Markovian  queue 
whose  rates  are  modulated  by  a  two-state  process  with  negative-exponential  inter¬ 
transition  intervals.  The  equilibrium  state  measures  were  computed  via  the  solution 
of  a  system  of  differential  equations  obtained,  in  part,  from  the  Kolmogorov  forward 
equations.  Though  Eisen  and  Tainiter  are  given  credit  by  Purdue  [88]  as  being  the 
first  to  incorporate  variable  arrival  and  service  rates  into  a  queueing  model,  there 
were  others  working  concurrently  on  related  issues.  The  queueing  model  described  by 
Avi-Itzhak  [18]  in  the  second  of  a  two-part  article  is  priority-based  with  heterogenous 
arrivals,  but  with  service  rates  chosen  at  random  (i.e.  according  to  a  probability 
distribution)  at  each  arrival.  This  system  also  included  server  breakdowns,  but 
customers  avoided  being  pre-empted  by  repeatedly  choosing  service  times  until  a 
sufficiently  short  service  interval  is  obtained.  This  paper  comes  closest  in  spirit  to 
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the  research  that  we  present  here,  namely  in  that  if  preemptions  were  allowed,  then 
the  system  becomes  a  retrial  system  in  a  random  environment  subject  to  breakdowns. 

The  interesting  feature  of  Leese  and  Boyd’s  work  [69]  is  that  they  present  nu¬ 
merical  techniques  for  the  computation  of  transient  measures  of  a  simple  M/M/1 
queue  in  which  the  service  rate  depends  explicitly  on  time.  On  the  other  hand, 
Yechiali  and  Naor  [115]  independently  arrive  at  the  same  results  for  the  same  two- 
state  model  as  in  Eisen  and  Tainiter’s  paper  in  [37],  after  which  Yechiali  generalized 
this  work  to  the  case  of  greater  than  two  states  in  [114],  Neuts  [79]  developed  tran¬ 
sient  as  well  as  steady-state  results  for  a  Markov-modulated  M/G/l  system  in  which 
service  rates  are  fixed  for  the  duration  of  service.  This  model  sets  the  paradigm  - 
namely  that  system  transitions  may  not  occur  simultaneously  -  that  governed  his 
later  work  [80]  and  the  Markov-modulated  queueing  systems  studied  by  other  re¬ 
searchers  in  recent  years.  Finally,  Purdue  [88]  incorporated  all  of  the  pioneering 
work  heretofore  mentioned  into  a  rigorous  analysis  of  the  M /M/1  queue  in  a  Marko¬ 
vian  environment  in  an  article  published  in  1974.  In  it,  he  derived  one  of  the  earliest 
results  for  the  busy  period4  when  the  Markovian  environment  possesses  greater  than 
two  states.  There  are  also  a  number  of  other  papers  during  this  timeframe  that 
deal  with  service-  or  interarrival-time  dependency  on  an  external  process;  see  [44] 
for  a  look  at  how  the  random-environment  mechanism  evolved  over  its  initial  20-year 
period. 

In  1976,  Kogan  and  Litvin  [56]  considered  finite-capacity  queues  in  a  two-state 
Markov  random  environment  and  derived  the  associated  stationary  measures  of  mean 
queue  length  and  the  probability  of  service  failure  via  transforms.  Although  the 
authors  intended  ‘service  failure’  to  mean  a  failure  in  throughput  (i.e.  probability  of 
a  full  queue),  this  was  the  first  time  that  anyone  had  ever  considered  the  probability 
of  an  undesired  event  in  random  environments.  In  1978,  Neuts  again  published 
two  articles  [84,  85]  concerning  random  environments  in  queueing.  The  first  in 

4This  denotes  the  contiguous  period  of  time  between  two  epochs  in  which  the  queue  is  empty. 
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the  series  focused  upon  the  same  type  of  M/Ad/1  queue  in  a  random  environment 
considered  in  [37].  This  time,  however,  he  considered  the  driving  Markov  chain  in 
the  framework  of  the  quasi-birth-and-death  process  (QBD)  and  then  applied  the 
matrix-analytic  treatment  to  compute  the  steady-state  measures  of  interest.  This 
paper  reflected,  among  other  things,  Nents’  predilection  for  methods  that  allow  for 
the  algorithmic  computation  of  state  measures  in  the  elegant  matrix-analytic  fashion. 
In  the  second  half  of  this  research  (i.e.,  the  second  article),  Nents  contributed  two 
important  results.  The  first  is  that  the  equilibrium  queue-length  distribution  at 
the  end  of  a  sojourn  is  the  same  as  that  any  time  during  the  sojourn.  The  second 
important  development  is  that  he  derived  steady-state  results  for  the  multi-server 
queue  Ad/Ad/c,  which  is  the  first  time  that  anyone  has  accomplished  any  analysis  for 
a  multi-server  queue  in  a  random  environment. 

During  the  latter  part  of  the  1970s,  a  good  degree  of  specialization  took  place  in 
the  literature  as  various  authors  began  to  explore  a  range  of  issues  that  were  already 
known  for  queues  with  constant  parameters.  Many,  if  not  the  majority  of  these 
earlier  works  were  done  with  a  type  of  single-server  queue  since  the  analyses  were 
rather  involved  even  for  the  M/M/1  in  a  random  environment.  In  1984,  Mokaddis, 
Elias,  and  Metwally  [77]  studied  the  M/M/1  bulk-service  system  whose  service  and 
arrival  rates  are  subject  to  modulation  by  a  bivariate  Poisson  process.  A  bulk-service 
system  is  one  in  which  an  idle  server  will  take  exactly  r  >  1  customers  from  the  queue 
and  serve  them  en  masse  according  to  a  randomly-assigned  service  time.  Mokaddis, 
Elias,  and  Metwally  [78]  likewise  considered  an  Ad/M/1  system,  but  this  time  with 
an  unreliable  server  in  a  random  environment.  They  derived  the  mean  queue  length 
and  probability  of  service  failure  using  a  partial-generating  function  technique. 

Some  authors  extended  their  considerations  to  queues  with  arbitrary  arrival 
and/or  service-time  distributions.  Baecelli  and  Makowski  [20]  in  1986  obtained  sta¬ 
bility  conditions  for  the  G/G/l  queue  with  service  subject  to  a  random  process. 
Their  conclusion  was  that  the  workload  in  queue  was  larger  for  the  corresponding 
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system  with  convex  ordering  than  for  one  with  a  deterministic  service  rate  based  on 
the  average  service  time.  Of  significant  notice  is  mention  of  the  intensity-conservation 
laws  for  queues  that  were  derived  by  Miyazawa  [76]  in  1985,  which  were  employed  in 
order  to  place  bounds  on  the  workload  process.  The  authors  also  mentioned  Rolski’s 
article  [93]  in  which  the  the  stability  conditions  are  derived  for  single-server  queues 
with  an  ergodically  stable  sequence  of  random  variables  (see  [27]  for  a  concise  defini¬ 
tion)  forming  the  nonstationary  input  process.  Rolski  then  proved  that  the  average 
waiting  time  is  greater  than  that  of  the  corresponding  M/G/l  queue  with  the  same 
arrival  and  service  intensities. 

Much  work  had  been  accomplished  for  modulating  processes  of  single-server 
queues  with  exponentially-distributed  input  and  service  times.  However,  a  number  of 
authors  did  consider  service  or  arrival  processes  that  were  a  deterministic  function 
of  the  states  of  the  modulating  process.  In  a  very  recent  article,  Mahabhashyam 
and  Gautam  [73]  studied  a  single-server  Poisson-fed  queue  with  infinite  capacity 
and  FIFO  discipline.  Of  significant  note  here  are  the  assumptions  that  (i)  if  the 
environment  process  is  a  continuous-time  Markov  chain  (CTMC)  {Z(t)  :  t  >  0}, 
then  the  service  is  performed  at  a  rate  of  bzn )  units  of  work  per  unit  time;  in 
other  words,  the  work  is  constant  between  transitions  of  the  modulating  CTMC, 
and  (ii)  state  transitions  of  the  CTMC  are  permitted  during  service.  They  derived 
the  first  and  second  moments  of  the  service  time  using  first-step  analysis,  with  the 
conclusion  that  the  moments  of  service  time  are  dependent  upon  the  arrival  process. 
In  order  to  compute  the  essential  measures  of  queue  length  and  waiting  time5  the 
authors  resorted  to  a  matrix-geometric  approach.  They  completed  their  discussion 
with  applications  to  networks  and  CPU  processor-sharing.  The  reader  is  referred  to 
[26,  91,  94,  109]  for  further  reading. 

5The  distributions  of  queue  length  and  waiting  time  cannot  be  obtained  in  closed  form  for  this 
system. 
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As  is  usually  the  case,  the  derivation  of  steady-state  measures  abounds  in  the 
literature  due  to  complexity  issues  that  plague  the  derivation  of  the  time-dependent, 
or  transient  measures  of  a  queue,  and  especially  one  that  is  modulated  by  an  external 
process.  In  1992,  Lee  and  Li  [68]  considered  a  Markov-modulated  Poisson-arrival 
queue  under  overload  control.  That  is,  when  the  buffer  content  exceeds  a  certain 
level,  the  arrival  process  is  modified  in  order  to  mitigate  any  overflow  condition. 
The  authors  determined  the  transient  distribution  of  queue  length  and  the  first 
passage  time  to  and  from  overload  status  for  this  model.  They  then  determined  the 
optimal  conditions  under  which  the  buffer  content  rises  as  slowly  as  possible  and 
decreases  as  quickly  as  possible.  Finally,  the  authors  studied  how  the  properties 
of  the  modulating  Markov  chain  affect  the  transient  behavior  with  regard  to  the 
aforementioned  state  measures. 

Markov-modulated  queues  using  the  matrix-geometric  approach  to  queueing 
developed  by  M.F.  Neuts  during  the  1970s  has  created  an  entirely  new  discipline  be¬ 
cause  of  the  fresh  approach  that  it  offers  to  existing  problems  that  are  intractible  un¬ 
der  analysis  by  conventional  means.  In  2005,  Mitrani  [75]  applied  matrix-geometric 
techniques  to  unbounded  queueing  systems  in  Markovian  environments.  His  stated 
goal  was  to  obtain  suitable  approximations  for  the  exact  state  measures  of  the  system 
under  a  heavy  load  and  subject  to  an  environment  with  many  states.  As  the  au¬ 
thor  explains,  existing  methods  for  computing  exact  measures  suffer  from  numerical 
instability  and  complexity  due  to  such  factors  as  ill-conditioned  matrix  terms.  The 
proposed  method  centered  around  a  derived  matrix  expression  Q(x)  and  a  proposi¬ 
tion  concerning  this  matrix.  In  simple  terms,  the  proposition  states  that  the  associ¬ 
ated  QBD  for  the  Markov-modulated  system  is  ergodic  if  and  only  if  the  number  of 
eigenvalues  of  Q(x )  in  the  unit  disk  is  equal  to  the  number  of  states  of  the  Markovian 
environment  process.  From  this,  one  can  conclude  that  the  joint  distribution  of  the 
queue  size  and  environment  is  close  to  being  geometrically  distributed,  with  param- 
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eter  identical  to  the  dominant  eigenvector  of  Q(x).  This,  of  course,  leads  naturally 
to  a  method  by  which  one  may  approximately  ascertain  the  queue  length. 

It  is  worth  mentioning  that  the  notion  of  fluid  queueing  was  being  developed  in 
conjunction  with  the  random  environment  concept.  In  fact,  fluid  queueing  owes  its 
existence  to  the  notion  of  a  random  environment  since  rates  of  input  flow  of  a  contin¬ 
uous  fluid  are  typically  modulated  by  an  external  stochastic  process.6  It  is  this  con¬ 
tinuous  input  that  distinguishes  the  fluid  system  from  the  standard  queueing  system. 
Nevertheless,  it  becomes  apparent  that  the  fluid  queue  may  be  (roughly)  analogized 
to  an  infinite- server  queueing  system  that  is  Markovian  if  its  rate-modulating  process 
is  likewise  Markovian.  Kella  and  Whitt  [50]  uncover  yet  another  analogy  to  standard 
queueing  in  considering  a  fluid  system,  and,  namely  one  with  a  type  of  compound 
modulating  process.  The  fluid  queue  alternates  between  ‘up’  and  ‘down’  states;  when 
the  system  is  up,  the  buffer  content  decreases  according  to  one  stochastic  process 
and  when  it  is  down,  the  buffer  content  increases  according  to  another  process.  The 
authors  then  demonstrated  that  the  steady-state  buffer  content  is  directly  related  to 
the  virtual  waiting  time  in  a  G/G/l  queue  under  certain  assumptions. 

Non-queueing  systems  in  random  environments  have  also  been  considered,  and 
although  these  types  of  environment-modulated  processes  are  not  the  focus  of  this 
research,  the  methods  that  they  use  to  derive  associated  performance  measures  are 
highly  relevant.  In  1981,  Bourgin  and  Cogburn  [24],  discussed  the  probability  of 
passage  into  a  closed  set  of  absorbing  states  for  a  Markov  chain  in  a  random  environ¬ 
ment.  Economou  [36]  took  a  much  less  esoteric  approach  in  deriving  the  stationary 
distributions  for  measures  of  a  bivariate  discrete  Markov  chain  {( En,  Xn)  :  n  >  0}, 
of  which  the  first  term  is  taken  from  the  random  environment  {En  :  n  >  0}.  A  year 
later,  Hu  [46]  considered  a  Markov  chain,  a  renewal  process,  and  a  random  walk  - 
as  well  as  a  queue  -  in  a  random  environment  and  established  each  of  their  state 
space  decompositions.  In  1992,  Korotaev  and  Spivak  [57]  considered  Unite-capacity 

6The  output  rate  is  usually  deterministic. 
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queues  with  parameters  that  vary  according  to  a  semi-Markov  process.  Moreover, 
the  system  state  changes  very  slowly,  with  sojourn  times  in  an  inverse-reciprocal 
relationship  with  a  small  positive  e.  The  authors  obtained  the  distribution  of  the 
number  in  the  system  via  solution  of  coefficients  in  a  series  expansion. 

Reliability  analysis  in  queueing  theory  is  a  natural  direction  for  the  research 
into  failure  models  to  take.  We  see  the  first  set  of  such  works  for  reliability  systems 
(mainly  pioneered  by  Sztrik;  see  [10,  101,  103,  104,  105,  106])  scheduling,  and  finan¬ 
cial  stochastic  systems  and  other  logistic  considerations  appear  in  this  context  from 
1989  onward;  see  [28,  33,  45,  47,  51,  86,  99]  for  specific  examples. 

Of  immediate  relevance  is  the  topic  of  Markov-dependent  single-server  queues; 
that  is,  those  queueing  systems  whose  exponential  service  and  arrival  processes  de¬ 
pend  upon  each  other.  The  literature  here  is  quite  extensive,  but  does  not  necessarily 
contain  works  in  which  the  inter-arrival  and  service  times  are  modulated ;  nor  do  they 
deal  with  server  breakdowns.  One  of  the  more  recent  works  in  this  area  is  by  Adan 
and  Kulkarni  [3],  who  study  the  limiting  distributions  of  the  waiting  time  and  queue 
length  of  a  semi-Markov  queue  whose  inter-arrival  and  service  times  are  modulated 
by  the  same  discrete-time  Markov  chain.  The  assumptions  are  no  more  stringent 
than  that  of  the  queue  possessing  the  Markov  property,  which  considerably  sim¬ 
plifies  analysis.  Hence,  the  importance  of  this  work  is  that  the  authors  consider 
single-server  Markovian  queues  with  general  distributions  and  subject  to  a  random 
environment. 

2.4  Unreliable  Queueing  Systems  in  Random  Environments 

There  does  not  exist  an  abundance  of  papers  in  the  literature  that  deal  with 
queues  in  random  environments  and  the  possibility  of  service  interruptions  and/or 
vacations,  at  least  explicitly.  The  idea  was  recognized  early,  as  Avi-Itzhak  did  in 
his  1963  articles  [17,  18],  but  obviously  presented  too  many  challenges  at  such  an 
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early  time  in  the  development  of  the  subject.  Regardless,  failures  are  implicit  in  any 
practical  model  of  a  real-world  system,  and  so  the  foundations  for  the  modulated 
queue  with  failure  were  established  almost  as  soon  as  the  first  articles  on  models 
with  process-dependent  parameters.  In  1963,  Avi-Itzhak  and  Naor  [19]  describe 
five  different  single-server  queues  that  incorporate  different  assumptions  about  the 
failure.  The  first  failure  model  is  based  upon  a  stationary  Poisson  process  with 
no  restrictions.  The  second  model  assumes  that  failures  can  occur  only  during  a 
busy  period,  the  third  assumes  that  failures  occur  when  the  system  is  nonempty, 
the  fourth  assumes  that  repair  takes  place  only  at  the  request  of  a  customer,  and 
the  last  assumes  that  failures  occur  only  during  idle  periods.  The  repair  and  and 
service  times  are  both  generally-distributed  with  density  and  finite  second- moments 
for  each  model. 

While  this  work  established  a  methodology  for  the  incorporation  of  failure  into 
a  model,  some  time  passed  before  these  were  found  in  conjunction  with  variable 
rates.  In  1976,  Kogan  and  Litvin  [56]  computed  the  asymptotic  measures  for  a 
queueing  system  in  an  unspecified  random  environment  and  subject  to  service  fail¬ 
ures.  Mokaddis,  Elias,  and  Metwally  [77,  78]  did  the  same  for  modified  M/M/1 
queues  with  Poisson-modulated  rates.  In  1999,  Kroese  and  Nicola  [59]  considered 
single-server  (fluid  and  discrete)  queues  with  alternating  failures  and  Markov  mod¬ 
ulation  of  the  Poisson  arrivals  and  generally-distributed  service  rate.  They  employ 
results  on  Markov-additive  processes  to  obtain  results  on  the  optimal  change  of 
measure ,  and  the  concept  of  the  effective  bandwidth  is  used  to  restrict  the  number 
of  environmental  states  that  need  to  be  included. 

Finally,  in  2005,  Klimenok  [54]  studied  what  is  essentially  the  first  known 
retrial  queueing  system  in  a  random  environment  to  appear  in  the  open  literature. 
The  model  is  comprised  of  a  single-server  with  a  batch  Markovian  arrival  process 
(BMAP)  and  semi-Markov  service  and  retrial  intervals.  The  process  that  defines  the 
random  environment  is  a  bivariate  Markov  chain  { (ry ,  st)  :  t  >  0}  with  a  finite  state 
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space.  The  random  variables  rjruSt  of  customers  served  sequentially  are  influenced 
by  the  random  environment  by  definining  them  to  be  geometric  with  parameter  qryt. 
The  first  term  of  the  bivariate  process  controls  a  hybrid  mechanism  in  which  the 
queue  is  determined  to  be  either  a  retrial  queue  or  a  ‘system  with  waiting’  based  on 
its  membership  in  a  partition  of  the  subsets  {(r,  •)}  of  the  state  space.  The  second 
term  controls  the  parameters  of  the  BMAP  input  and  the  semi-Markov  intensities  of 
service  and  retrial.  It  is  a  synchronous  random  environment  in  the  sense  that  it  only 
changes  its  state  at  service  completions,  thus  obviating  the  need  to  consider  changes 
in  the  service  rate  during  transitions  of  the  random  environment.  The  author  then 
employed  an  embedded  Markov  chain  to  evaluate  the  queue  in  steady-state,  and 
thus,  derive  the  associated  distributions  of  state  measures  as  probability  generating 
functions. 

Aside  from  [54],  the  literature  concerning  Markov-modulated  retrial  queueing 
systems  is,  at  best,  sparse,  and  such  a  model  that  includes  failures  of  the  server(s) 
does  not  exist  to  the  author’s  knowledge.  Thus,  it  is  the  aim  of  this  research  to 
supplement  the  retrial  queueing  literature  with  novel  insights  into  the  stability  and 
steady-state  behavior  of  a  class  of  models  that  has  not  been  considered  previously, 
namely  the  M/M/ 1  and  the  M/G/l  versions  of  the  unreliable  retrial  queueing  system 
in  a  random  environment.  Moreover,  it  is  crucial  that  the  analysis  presented  here  be 
useful  for  practical  application,  which  suggests  that  our  approach  must  be  oriented 
to  computational  considerations  and  algorithmic  development.  The  matrix- analytic 
theory  turns  out  to  be  an  ideal  framework  for  this  purpose.  It  is  firmly  grounded 
in  mathematical  principles,  and,  yet,  is  easily  utilized  in  the  computational  investi¬ 
gation  of  queueing  performance.  In  this  dissertation,  we  seek  to  not  only  apply  the 
matrix-analytic  methods,  but  also  to  extend  their  applicability  to  the  larger  classes 
of  level-dependent  GI/M/ 1  and  M/G/  1-type  systems. 
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3.  Preliminaries 

Markov  chains  play  a  fundamental  role  in  the  theory  of  queues,  and  particularly 
those  that  can  be  categorized  as  birth- and- death  processes.  These  are  continous- 
tirne  Markov  chains  (CTMCs)  for  whom  transitions  are  allowed  only  to  neighboring 
states.  Suppose  that  we  are  given  a  homogeneous  CTMC  -pf(i)  :  t  >  0},  where  X(t) 
denotes  the  population  at  time  t.  The  state  space  of  this  CTMC  is  S  =  Z+,  which 
are  the  nonnegative  integers,  and  Q  =  [qtJ]  is  its  infinitesimal  generator  .  When  the 
population  is  i,  that  is  X(t)  =  i,  then  the  exponential  rate  of  births  is  A,;  and  the 
rate  of  deaths  is  /q.  In  mathematical  terms,  this  translates  to 


Qi,i+ 1 

^2} 

if  i  >  0 

Qi,i—  1 

A t'i') 

if  i  >  1 

Qij 

=  0  otherwise 

The  transition  rate  diagram  for  a  birth-and-death  process  is  shown  in  Figure  3.1.  As 
a  consequence  of  the  definition  of  the  transition  rates,  we  may  write  the  generator 
Q  in  the  following  manner: 


^0  ^n-1  An+1 


Figure  3.1  Transition  rate  diagram  for  a  standard  birth-and-death  pro¬ 
cess. 
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Let  pj  be  the  probability  that  j  G  Z+  is  the  population  of  the  system  at  steady- 
state.  The  existence  of  the  steady  state  distribution  for  a  birth-death  system  hinges 
upon  the  existence  of  a  solution  to  the  system  of  equations 

pQ  =  0,  pe  =  1, 


where  A*  >  0,  Hi  >  0,  i  G  Z+,  and  e  is  a  row  vector  containing  ones.  If  a  solution 
exists,  it  is  given  by 


Po  = 


OO 


(3.1) 


Pi  =  vhpo,  i  >  1, 


(3.2) 


where 


7L  = 


£— 1 


n 


h'i+i 


i  >  1. 


The  product  form  (3.2)  of  the  steady-state  probabilities,  in  particular,  is  a  hallmark 
of  all  birth-and-death  processes.  This  relationship  to  state  0  is  of  fundamental  im¬ 
portance  to  these  and  other  more  generally-defined  quasi-birth-and-death  (QBD) 
processes  that  we  shall  discuss  next,  and  is  the  cornerstone  of  the  matrix- analytic 
approach  as  it  pertains  to  these  models. 
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Many  Markovian  queueing  systems  are  modeled  as  birth-  and-death  processes. 
For  example,  the  MjMjc  and  M/M/c/c  queueing  systems  evolve  as  birth-and-death 
processes.  However,  the  simple  birth-and-death  model  fails  to  account  for  multiple 
interacting  stochastic  processes  that  might  coexist  in  more  complex  systems,  and 
thus,  a  more  general  paradigm  has  become  prevalent  in  the  queueing  community.  In 
this  chapter,  we  review  the  rudimentary  notion  of  the  QBD  and  illustrate  the  tech¬ 
niques  with  which  one  may  easily  analyze  such  stochastic  processes.  Subsequently, 
we  discuss  the  extension  of  this  idea  to  non-Markovian  processes  that  behave  in  a 
similar  manner,  but  nevertheless  exhibit  non-exponential  random  behavior.  These 
preliminary  concepts  are  needed  for  the  formal  analysis  of  unreliable  retrial  queues 
that  operate  in  a  randomly  evolving  environment. 

3.1  Quasi- Birth- and- Death  (QBD)  Processes 

The  study  of  quasi-birth-and-death  processes  is  inextricably  tied  to  the  matrix- 
analytic  approach  of  Neuts  (see  [81,  82,  83]).  In  contrast  to  the  more  conventional 
^-transform  approach  used  to  compute  the  limiting  distribution  of  such  processes, 
the  matrix-analytic  method  takes  advantage  of  a  common  structure  that  is  shared 
amongst  a  wide  variety  of  seemingly  unrelated  models.  Such  general  applicability 
leads  to  an  algorithmic  approach.  While  this  may  seem  to  be  a  trade  off,  it  soon 
becomes  apparent  that  the  method  allows  for  the  solution  of  realistic  systems  with 
significant  levels  of  complexity. 

Such  tractability  is  a  consequence  of  the  fact  that  essential  quantities,  such  as 
those  that  exist  in  (3.2),  may  be  represented  as  single  matrix  entities  regardless  of 
the  size  or  complexity  of  the  model.  This  fact,  when  combined  with  the  convenient 
matrix- geometric  distribution  of  the  steady-state  probabilities,  such  as  that  which  is 
embodied  in  (3.2),  allows  for  the  relative  ease  of  the  computation  of  the  steady-state 
performance  measures  for  models  that  may  be  classified  as  QBDs. 
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We  shall  next  review  basic  definitions,  concepts,  and  techniques  of  the  theory 
of  quasi-birth-and-death  (QBD)  processes,  all  of  which  is  founded  upon  the  matrix- 
geometric  property  of  discrete-  and  continuous-time  Markov  chains.  We  shall  initially 
present  basic  concepts  that  fall  within  the  framework  of  discrete-time  QBDs.  This  is 
done  for  two  reasons.  First  is  the  fact  that  the  properties  of  continuous-time  QBDs 
are  derived  from  those  of  their  discrete-time  embedded  Markov  chains,  which  are 
likewise  QBDs.  Second,  probabilistic  interpretations  are  more  clearly  understood 
and  applied  for  the  discrete  case  by  virtue  of  the  fact  that  these  are  defined  by 
their  transition  probabilities.  The  important  distinction  is  that  which  separates 
level-independent  QBDs  from  those  that  are  level-dependent,  the  latter  type  being 
not  only  more  complex,  but  which  also  require  a  different  set  of  methods  for  their 
steady-state  analysis. 


3.1.1  Level-Independent  Processes 

A  bivariate  DTMC  {(Xn,Yn)  :  n  >  0}  is  a  QBD  if  its  transition  probability 
matrix  appears  in  block-tridiago7ial  form 


A\  Aq  0  0  0 

A2  Ai  Aq  0  0 

0  A. 2  A\  Aq  0 

0  0  A 2  A\  Aq 

0  0  0  A2  A\ 


(3.3) 


where  Aq,  A\,  and  A2  are  square  matrices  whose  dimensions  are  determined  by  the 
structural  properties  of  the  system  at  hand.  We  further  observe  that  the  off-diagonal 
blocks  Aq  and  A2  contain  only  positive  entries  as  they  provide  information  about  the 
transition  probabilities  (or  rates  in  the  continuous-time  model)  of  the  process.  Each 
row  of  P  is  termed  a  level,  which  is  comprised  of  related  system  states  called  phases. 
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It  is  possible  to  transition  in  one  step  either  to  adjacent  levels  or  to  any  other  phase 
within  the  same  level,  hence  the  term  ‘quasi-birth-and-death’.  Incidentally,  the  size 
of  the  QBD  blocks  A} ,  j  =  0, 1,  2  corresponds  to  the  number  of  phases  per  level.  For 
example,  if  the  block  size  isMxM  where  M  is  a  positive  integer,  then  there  are 
M 2  phases,  and  vice-versa. 

A  QBD  is  termed  level-independent  or  homogeneous  if  it  exhibits  an  infinitely 
repeating  tridiagonal  block  structure  such  as  the  one  represented  in  (3.3).  Never¬ 
theless,  it  is  usually  the  case  that  the  states  that  comprise  level  0  may  differ  from 
those  at  any  other  level.  We  therefore  refer  to  these  states  as  the  boundary  states 
<S(0)  C  S,  where  S  is  the  state  space  of  the  entire  QBD.  A  crucial  point  to  be  made 
here  is  that  the  process 

{(A'(«»,K„(«»):n>0}  (3.4) 

restricted  to  the  boundary  states  at  level  0  (otherwise  known  as  the  process  at 
level  0  under  taboo  of  all  other  levels  of  the  QBD)  is  itself  a  Markov  chain  with 
invariant  probability  vector  tzq  and  its  own  transition  probability  matrix  (to  be 
defined  later).  The  importance  of  the  process  restricted  to  level  0  is  derived  from  its 
role  in  determining  the  matrix-geometric  distribution  of  the  steady-state  distribution 
of  the  overall  QBD,  as  hinted  in  the  preface  to  this  chapter. 

The  discussion  that  follows  is  drawn  from  [66],  which  consequently  may  be 
referenced  for  proofs  and  other  details  that  we  do  not  present  here.  We  begin  with 
the  assumption  of  homogeneity.  In  order  to  adequately  convey  what  is  meant  by 
the  term  ‘matrix  geometric’,  we  define  the  following.  Let  {( Xn,Yn )  :  n  >  0}  be 
a  discrete-time  level-independent  QBD  with  transition  probability  matrix  P  and 
a  finite  number  of  phases  N  >  0.  We  define  the  steady-state  joint  probabilities 
7 Ty,  i,j  G  Z+  to  be  the  limiting  values 

n.ij  =  lim  P  {Xn  =  i,  Yn  =  j}. 

n— xx) 
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Next,  define  7r  to  be  the  row  vector  \pij\iJez+  such  that  7r  =  [7r00,  7t01,  . . . ,  7 r10, 7rn, . . .]. 
It  is  a  well-known  fact  that  7r,  if  it  exists,  is  the  unique  solution  to  the  system  of 
equations  given  by 

7 tP  =  7T ,  7re  =  1, 
where  e  is  a  column  vector  of  ones. 

We  partition  this  vector  according  to  levels  of  the  QBD,  which  is  to  say  that  we 
set  7r  =  [tt0,  77 1 ,  7r 2 , . . .],  where  7r,  =  [77* 0, . . . ,  7Tjjv]  for  i  >  0.  The  matrix  geometric 
property  of  the  steady  state  distribution  is  then  expressed  via  the  recurrence  relation 

7 Ti+i  =  tt0Rl,  (3.5) 

where  R  is  a  matrix  such  that,  for  each  i  >  0,  Rjk  (0  <  j,  k  <  N)  records  the 
expected  number  of  visits  to  state  (i  +  l,k)  before  returning  to  level  i,  given  that 
the  process  starts  in  (i,j).  The  matrix  R  is  often  referred  to  as  the  rate  matrix  since 
it  may  be  interpreted  as  the  ‘rate  of  visit’  to  level  i  +  1 . 

It  is  apparent  from  (3.5)  that  determining  77o  and  R  is  key  to  the  computation 
of  the  steady  state  probabilities  for  a  given  discrete-time  QBD.  It  can  be  shown  that 
the  restricted  process  (3.4)  of  the  QBD  is  a  positive  recurrent  Markov  chain  if,  and 
only  if,  the  overall  QBD  process  is  likewise  positive  recurrent  (see  [66:  Thm  5.3.1]). 
The  restricted  process  (3.4)  possesses  the  transition  probability  matrix  B  +  AqG  and 
the  corresponding  steady  state  vector  7T0.  Consequently,  the  invariant  probability 
vector  7Tq  may  be  determined  in  the  usual  way  as  the  unique  positive  solution  to  a 
linear  system  of  equations  as  described  in  Theorem  3.1. 

Theorem  3.1.  The  vector  7r0  is  the  unique  steady  state  solution  to  the  system  given 
by 


7ro(-S  +  AqG)  —  7Tq, 

(3.6) 

3 

O 

- - - 

1 

1 — 1 

(3.7) 
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Notice  that  solving  the  system  (3.6)  -  (3.7)  requires  explicit  determination  of  the 
rate  matrix  R.  One  might  attempt  to  solve  the  following  matrix-quadratic  equation 
to  obtain  R ,  although  this  is  not  practical  due  to  the  computational  burden. 

Theorem  3.2.  The  rate  matrix  R  is  the  minimal  (positive)  and  unique  solution  to 
the  matrix  quadratic  equation 


Aq  +  RA\  +  R~A2- 


It  is  clear  that  this  criterion  may  fail  to  be  of  practical  use  for  all  QBDs,  save 
those  that  possess  the  smallest  of  block  matrix  orders.  Hence,  we  must  resort  to  an 
algorithmic  approach  in  order  to  obtain  the  rate  matrix.  To  this  end,  we  will  now 
define  two  other  fundamental  matrices. 

The  matrices  U  and  G  that  we  shall  now  define  are  important  to  the  character¬ 
ization  of  the  steady-state  distribution,  and  hence,  to  the  development  of  algorithms 
to  find  the  rate  matrix  R.  The  th  element  of  the  matrix  G  is  given  by 

Gj?  =  P{t  <  oo,  XT  =  (i,f)  |  X0  =  (i  +  1  ,j) ),  (3.8) 

where  r  is  a  discrete  random  variable  defined  as  the  first  passage  time  from  level  %  + 1 
to  level  i.  It  may  also  be  determined  exactly  as  the  solution  to  the  matrix  quadratic 
equation 

A2  +  A\G  +  AqG~  =  0,  (3.9) 

although,  just  as  with  (3.2),  computational  issues  make  this  approach  impractical. 
The  other  matrix,  namely  U  =  Ai  +  A0G,  may  be  interpreted  as  being  the  transition 
matrix  of  the  QBD  process  restricted  to  level  i  until  the  first  visit  to  %  —  1.  The 
matrices  R ,  U,  and  G  can  each  be  deduced  from  any  of  the  others  via  the  following 
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mathematical  relationships: 


u  = 

Ai  +  RA2, 

(3.10) 

R  = 

A0(-U)-\ 

(3.11) 

G  = 

{-U)-lA2. 

(3.12) 

Though  rarely  used  to  explicitly  prove  the  positive  recurrence  of  a  QBD,  Theorem 
3.3  is  of  significant  import  to  theoretical  as  well  as  computational  considerations. 

Theorem  3.3.  The  QBD  process  given  by  {(Xn,Yn)  :  n  >  0}  is  positive  recurrent 
if  and  only  if  the  matrix  G  is  stochastic. 

Proof.  If  the  matrix  G  were  substochastic,  then  it  must  be  true  that,  for  some  j  <  N, 
we  must  have 

N  N 

=  Ep(r<°°’x-  =  (1~1g)\Xo  =  M)<  1, 

2—1  2—1 

which  shows  that,  at  best,  the  QBD  is  null  recurrent.  On  the  other  hand,  positive 
recurrence  requires  that  all  such  terms  must  equal  unity,  by  definition  of  the  matrix 
G.  □ 

The  stochasticity  of  G  can  often  be  used  as  a  convenient  stopping  criterion  for 
numerical  algorithms  used  to  approximate  the  steady-state  distribution  of  a  QBD. 
One  may  then  determine  the  rate  matrix  R  through  the  employment  of  equations 
(3.10-3.12). 

3.1.2  Level-Dependent  Processes 

A  natural  generalization  of  the  discrete-time  QBD  model  is  to  allow  the  blocks 
of  P  to  vary  according  to  level.  A  QBD  with  such  a  characteristic  is  called  level- 
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dependent  or  nonhomogeneous ,  with  a  transition  probability  matrix  represented  by 


4°}  A^  0  0  0 

A{4)  41}  4^  0  0 

o  42>  42)  42)  o 

0  0  43)  43)  a(03) 

0  0  0  a(24)  a{4) 


(3.13) 


We  may  further  relax  the  requirement  that  the  off-diagonal  blocks  be  square  matrices, 
or  even  that  there  must  exist  only  a  finite  number  of  phases  per  level.  Examples 
of  systems  with  level-dependent  QBD  representations  abound  in  the  literature.  The 
standard  Markovian  single-server  retrial  queue  with  independent  retrials  and  no 
buffer,  for  instance,  has  a  level-dependent  QBD  representation  that  comes  about 
due  to  the  dependence  of  the  total  retrial  rate  upon  the  level,  or  number  of  entities 
attempting  to  reaccess  the  server. 

It  so  happens  that  many  basic  facts  concerning  the  steady-state  distribution 
of  level- in  dependent  QBDs  may  be  extended  to  the  level-dependent  case  (see  [66:  p 
262]).  For  other  quantities  that  have  resisted  translation  from  the  level-independent 
case,  such  as  a  simple  drift  criterion,  we  may  still  resort  to  limiting  processes  taken 
over  levels  of  the  QBD.  Such  methods  are  straightforward  enough  in  practice,  but 
have  proven  to  be  quite  difficult  to  make  rigorous. 

The  classical  definition  of  QBDs  has  recently  evolved  to  include  Markov  chains 
{(X^,X^,...,X^f)  :  n  >  0}  with  three  or  more  state  variables  whose  transition 
probability  matrices  assume  the  tridiagonal  form.  Choose  one  of  the  component 
random  variables  with  an  infinite  state  space,  say  X„,  and  associate  each  of  its  pos¬ 
sible  states  with  levels  of  the  QBD.  This  transforms  the  above  multivariate  process 
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into  the  group-bivariate  process, 


which  in  turn  allows  the  transition  matrix  to  be  represented  in  the  usual  manner  by 
a  two-dimensional  matrix  in  tridiagonal  form.  In  this  research,  we  will  focus  upon 
what  is  called  in  [100]  a  tri-layered  QBD,  or  a  QBD  representation  of  a  stochastic 
system  with  three  state  variables.  The  topic  of  multi-layered  QBDs  has  only  recently 
begun  to  receive  much  attention,  as  the  original  definition  of  the  QBD  encompassed 
tridiagonal-structured  Markov  chains  of  only  two  state  variables,  the  first  of  which  is 
infinite- dimensional  (the  ‘level’)  and  the  other,  finite  (the  ‘phase’).  It  became  clear 
that  the  traditional  QBD  did  not  adequately  fulfill  every  stochastic  modeling  task, 
and,  moreover,  that  limiting  the  scope  of  QBD  processes  to  only  two- dimensions  is 
rather  restrictive. 

An  adequate  understanding  of  the  structure  of  a  multi-layered  QBD  is  crucial 
to  the  task  of  evaluating  the  steady-state  performance  of  the  system  under  consid¬ 
eration.  Each  so-called  layer  of  the  tri-layered  QBD  holds  a  particular  significance 
to  its  analysis:  the  lowest  (scalar-entry)  level,  which  we  shall  denote  here  by  level  2, 
is  used  in  the  numerical  computation  of  various  system  measures.  The  middle  layer, 
or  level  1  turns  out  to  be  critical  to  the  determination  of  a  closed-form  stability 
expression.  The  topmost  layer,  level  0,  should  exhibit  the  tridiagonal  form  if  the 
system  is,  indeed,  a  QBD. 

Regardless  of  the  number  of  layers  involved,  the  matrix-geometric  property  for 
(discrete-time)  level- dependent  QBDs  is  similar  to  that  for  level-independent  QBDs, 
with  the  exception  that  there  is  now  a  rate  matrix  that  corresponds  to  each  level 
i  >  1.  The  entries  of  the  matrix  gjVe  the  expected  number  of  visits  to  level  i 
between  successive  visits  to  level  i  —  1.  The  vector  7r  of  equilibrium  probabilities  in 
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this  case  satisfies  the  recurrence  relation 

7 Tj+1  =  7 TiR^+1\  i>  0.  (3-14) 

Accordingly,  each  level  i  >  0  is  associated  to  distinct  U®  and  matrices  that 
obey  relationships  similar  to  those  given  in  (3.10)-  (3.12).  The  level-dependent 
counterparts  to  the  matrix  quadratic  equations  (3.2)  and  (3.9)  are 

+  R(i+1)A[i+1)  +R(i+1)R(i+2)Aii+2),  (3.15) 

G(i+1)  =  A%+1) +  A(1i+1)G(i+1) +  A%+1)G(i+2)G(i+1).  (3.16) 

As  with  level-independent  QBDs,  the  positive-recurrence  of  a  level-dependent 
QBD  is  contingent  upon  the  fulfillment  of  certain  conditions  by  the  process  restricted 
to  level  0.  Theorem  3.4,  which  appears  in  [25,  66],  details  these  requirements,  as  well 
as  the  subsequent  matrix-geometric  relationship  between  the  resulting  steady-state 
probabilities. 

Theorem  3.4.  A  continuous-time  level- dependent  QBD  is  positive  recurrent  if  and 
only  if  there  exists  a  positive  solution  to  the  system  of  equations 

7T0  (40)  +  40)G(1))  =  0,  (3.17) 

TTo^Z  n  Rh'e  =  1  (3-18) 

i> 0  1  <k<i 

In  this  case,  the  steady-state  probability  vector  tt  =  [7T0,  7Ti,  . . .]  is  given  by 

i— 1 

iti  =  TT0Y\_Rk,  i>  0.  (3.19) 

k= 0 

Thus,  we  see  that  the  fundamental  results  for  level-dependent  QBDs  have  close 
analogues  to  those  already  derived  for  the  level-independent  case.  Nevertheless,  the 
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non-homogeneity  of  the  QBD  over  its  levels  contributes  significant  complexity  to 
algorithms  devoted  to  the  task  of  determining  the  equilibrium  distribution  of  the 
QBD.  We  shall  review  these  algorithms  in  greater  detail  in  Chapter  4. 

3.1.3  Ergodicity  of  QBDs 

In  this  section  we  review  the  conditions  that  guarantee  ergodicity  of  a  QBD, 
beginning  with  a  discussion  of  the  requirements  for  a  general  DTMC  over  a  countable 
state  space.  The  necessary  and  sufficient  condition  for  the  ergodicity  of  a  DTMC 
{Xn  :  n  >  0}  with  countable  state  space  S  is  that  the  process  must  be  aperiodic  and 
positive  recurrent.  We  further  assume  that  the  QBD  is  irreducible  or,  equivalently, 
that  each  state  is  reachable  from  all  others  via  a  finite  number  of  transitions,  a 
characteristic  that  greatly  simplifies  the  theoretical  considerations  that  attend  the 
steady-state  analysis. 

Positive  recurrence  is  a  class  property,  which  means  that  we  shall  define  pos¬ 
itive  recurrence  for  a  state  i  £  S.  Let  Si  be  the  time  of  the  first  transition  of  the 
QBD  process  and  let  r,;  =  min  {n  >  Si  \  Xn  =  i }  be  the  time  of  first  passage  into  the 
state  i.  The  state  i  is  then  called  positive  recurrent  if  the  following  conditions  hold: 

(i)  P(ji  <  oo  |  X0  =  i)  =  1,  (3.20) 

(ii)  E[ri\X0  —  i\  <  oo.  (3-21) 

A  communicating  class  of  a  Markov  chain  is  called  positive  recurrent  if  every  member 
of  that  class  is  so  designated;  thus,  for  an  irreducible,  positive  recurrent  Markov 
chain,  every  state  must  be  positive  recurrent.  The  definition  of  positive  recurrence 
for  a  CTMC  is  analogous  to  that  of  the  DTMC,  with  ‘max’  replaced  by  ‘inf’  in  the 
definition  of  the  first  passage  time  t%.  It  is  worthwhile  to  note  here  that  positive 
recurrence  is  not  necessarily  shared  between  a  CTMC  and  its  embedded  Markov 
chain  unless  it  possesses  a  finite  number  of  phases  per  level.  Since  we  will  analyze 
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such  a  finite-phase  model,  one  may  assume  henceforth  that  the  positive  recurrence 
of  one  implies  the  other. 

A  main  focus  of  this  research  is  to  determine  the  long-run  performance  mea¬ 
sures  of  a  system  that  is  ergodic.  For  the  finite-phase,  continuous-time  QBD,  the 
proof  of  its  ergodicity  is  equivalent  to  proving  that  its  embedded  chain  is  likewise 
ergodic,  which,  in  turn,  is  equivalent  to  proving  that  the  embedded  chain  is  positive 
recurrent  (since  we  assume  aperiodicity).  Note  that  the  determination  of  ergodic¬ 
ity  is  not  contingent  on  the  proof  of  the  positive  recurrence  of  every  state  of  the 
embedded  DTMC.  This  is  instead  a  consequence  of  the  ergodicity  property  for  an  ir¬ 
reducible  and  aperiodic  Markov  chain,  which  may  be  characterized  as  the  geometric 
convergence  of  the  transient  probabilities  of  the  transition  probability  matrix  (TPM) 
to  the  steady-state  probabilities;  in  other  words,  if  P  is  the  TPM  and  there  exists  an 
invariant  probability  vector  7r  of  the  chain,  and,  given  the  column  vector  e  of  ones, 

Pn  ->  eir 

converges  at  a  geometric  rate,  then  the  Markov  chain  is  deemed  ergodic.  This  fact 
is  presented  as  Theorem  1.1  of  [74], 

For  Markov  chains  with  a  countable  state  space,  the  drift  at  state  i  is  defined 
as  the  quantity  E[Xn+1  —  Xn  \  Xn  =  ?'],  where  i,  n  e  Z+.  A  positive  drift  at  i  implies 
that  the  process  will  tend  towards  higher-numbered  states  when  in  state  i,  while  a 
negative  drift  will  imply  exactly  the  opposite.  The  fundamental  results  that  leverage 
this  concept  of  drift  in  order  to  assert  the  ergodicity  of  a  Markov  chain  are  Foster’s 
criterion  for  stability  [61]  and  Pakes’  Lemma  [87].  Pakes’  lemma,  in  particular, 
is  well-known  by  virtue  of  its  being  the  first  simple  criterion  for  determining  if  a 
discrete-time  Markov  chain  with  a  countable  state  space  is  positive  recurrent.  Simply 
stated,  it  asserts  the  positive  recurrence  of  a  system  that  possesses  negative  drift  over 
a  subset  of  its  states.  Let  S  be  the  countable  state  space  of  a  DTMC  denoted  by 
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{Xn  :  n  >  0},  and  let  v  :  S  — >  [0,  oo).  A  related  quantity  called  the  generalized  drift 
at  state  i  of  a  DTMC  is  an  average  of  weighted  increments  v(Xn+i)  —  z/(Xn),  n  >  0, 
where  z/(-)  is  a  nonnegative,  real-valued  function  on  Z+.  It  is  formally  defined  for  a 
given  state  i  as 


d(i)  =  E[v(Xn+1)-v(Xn)\Xn  =  i] 

=  ^Pljn{j)-v(i),  ieS.  (3.22) 

j&s 

The  function  u{i)  is  commonly  termed  a  Lyapunov  or  potential  function  (see  [41]), 
which  is  a  measure  of  some  positive  quantity,  or  ‘potential’,  associated  to  a  particular 
state  of  a  process  with  a  countable  state  space.  Thus,  the  drift  may  be  equated  to 
the  notion  of  a  change  in  potential.  Foster’s  criterion,  which  we  shall  now  state, 
establishes  the  intuitive  fact  that  a  negative  net  potential  means  that  the  system 
will  not  traverse  its  state  space  in  an  unbounded  manner. 

Theorem  3.5.  (Foster’s  Criterion)  Let  [Xn  :  n  >  0}  be  an  irreducible  DTMC  on  a 
countable  state  space  S .  If  there  exists  a  function  v  :  S  — »  [0,  oo),  an  e  >  0,  and  a 
finite  set  H  C  S  such  that 


\d(i)\  <  oo  for  i  e  H,  (3.23) 

d(i)  <  -£  for  i  <£  H,  (3.24) 

then  {Xn  :  n  >  0}  is  positive  recurrent. 

The  form  of  d(i)  that  is  most  relevant  to  our  discussion  of  ergodicity  of  QBDs 
is  u(i)  =  i.  Indeed,  Pakes’  lemma,  which  we  shall  next  present,  may  be  considered 
as  a  special  case  of  Foster’s  criterion  in  which  the  Lyupanov  potential  is  the  identity 
function. 
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Theorem  3.6.  (Pakes’  Lemma).  Let  the  DTMC  {Xn  :  n  >  0}  defined  on  the  state 
spaceS  =  {0,1,2,...}  be  irreducible,  aperiodic,  and  homogeneous.  Let 

d(i)  =  E[Xn+ 1  —  Xn  j  Xn  =  i],  i  E  S 

and  suppose  that  the  following  conditions  hold  for  each  i  G  S: 

(i) .  d(i)  <  oo, 

(ii) .  lim  sup  d(i)  <  0. 

i—>  oo 

Then  {Xn  :  n  >  0}  is  ergodic. 

In  the  next  chapter  we  employ  Pakes’  lemma  to  describe  the  sufficient  conditions 
for  the  stability  of  level-independent  QBDs,  where  the  drift  is  the  measure  of  the 
average  increment  of  a  random  walk  over  the  state  space  of  a  QBD. 

Pake’s  lemma,  as  well  as  Foster’s  criterion,  is  limited  in  that  it  is  only  an 
expression  of  the  sufficiency  of  a  negative  drift.  In  [95]  there  appear  a  number  of 
results  that  provide  sufficient  conditions  for  the  non-ergo dicity  of  a  discrete-time 
Markov  chain,  which  means  the  contrapositive  argument  may  be  used  to  obtain  the 
necessary  conditions  for  its  ergodicity.  Define  ptJ  as  the  (i,j)th  component  of  the 
transition  probability  matrix  P  of  a  generic  DTMC  {Xn  :  n  >  0}  with  a  state  space 
equivalent  to  the  nonnegative  integers.  It  can  be  shown  that  the  non-ergodicity  of 
such  a  Markov  chain  is  predicated  upon  demonstrating  that  the  function 

ipi(z)  =  ^z1  —  y ^pijZJ^  /(I  —  z),  i  >  0  and  z  G  [0, 1),  (3.25) 

is  bounded  from  below  by  zero  in  some  interval  [c,  1)  C  [0, 1)  and  for  some  0  <  IV  < 
i.  The  requirement  on  the  value  of  z  is  guaranteed  by  the  fulfillment  of  Kaplan’s 
Condition ,  which  is  that  there  exists  an  £  >  0,  a  positive  integer  N,  and  c  G  [0, 1) 
such  that  'ifi(z)  >  —  £  for  i  >  N  and  z  G  [c,  1).  Furthermore,  the  connection  of  f>i(z) 
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to  drift  is  evident  in  the  observation  that 


d(i)  =  lim  if \{z ). 

Z—>1~ 

The  preceding  observations  culminate  in  the  result  that  is  Theorem  3.7,  which 
first  appeared  in  [95:  Thm  1]: 

Theorem  3.7.  If  the  discrete-time  Markov  chain  {Xn  :  n  >  0}  satisfies  Kaplan’s 
Condition,  d(i)  <  oo  for  all  i  >  0,  and  there  exists  a  positive  number  N  such  that 
d(i)  >  0  for  all  i  >  N,  then  {Xn  :  n  >  0}  is  not  ergodic. 

A  direct  proof  of  the  applicability  of  Kaplan’s  condition  may  be  difficult,  and 
it  is  thus  desirable  to  have  alternative  criteria  on  hand.  The  following  theorem  from 
[95]  provides  the  contextual  framework  in  which  the  condition  holds: 

Theorem  3.8.  Let  P  =  [pij\  be  the  transition  probability  matrix  of  a  DTMC  {Yn  : 
n  >  0}  and  define  the  sequences 

5i  =  -<) 

j<i 

Si  =  Pi3{j  -  i) 

j>i 

7 i  =  $i  +  £i- 

Next,  consider  the  following  statements: 

1.  There  exists  a  k  such  that,  for  j  <  i  —  k  and  i  >  0 ,  pij  =  0 . 

2.  The  sequence  {5*  :  i  =  0, 1,2,...},  is  bounded  frorn  below. 

3.  Kaplan’s  condition  holds. 

4-  The  sequence  {7 i  :  i  —  0, 1,2, . . .}  is  bounded  from  below  and  limwoc ptJ  =  0, 
where  j  >  0. 

Then  (1)  =>-  (2)  (3)  (4). 
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Condition  (2),  in  particular,  is  well-suited  to  the  task  of  demonstrating  Kaplan’s 
condition  for  processes,  such  as  those  of  M/G/  1-type,  that  allow  any  number  of 
positive  increments  (e.g.,  in  system  size),  but  only  a  limited  number  of  decrements. 

We  may  now  proceed  to  discuss  the  conditions  that  establish  the  ergodicity 
of  level-independent  QBDs.  We  note  here  that  for  the  level- independent  case,  it  is 
sufficient  to  characterize  the  ergodicity  of  a  QBD  in  terms  of  its  transitions  between 
levels ,  since  the  phase  space  is  identical  for  any  given  level.  Thus,  if  one  can  prove 
the  existence  of  an  embedded  Markov  chain  whose  state  space  consists  of  the  integers 
(and,  therefore,  the  levels  of  the  QBD)  and  whose  behavior  over  levels  is  exactly  that 
of  the  underlying  QBD,  we  may  then  apply  the  preceding  theorems  for  the  ergodicity 
of  a  Markov  chain. 

The  process  that  records  transitions  between  levels  bears  a  strong  resemblance 
to  a  one-dimensional  random  walk  over  the  nonnegative  integers  with  discrete  (inde¬ 
pendent)  steps,  or  increments  taken  in  the  positive  or  negative  directions.  We  shall 
briefly  discuss  the  development  of  this  idea  for  a  discrete-time,  level-independent 
QBD,  based  upon  the  presentation  in  [66:  Thm  5.3.1].  Suppose  that  such  a  QBD 
has  the  transition  probability  matrix  (3.3).  We  begin  by  defining  the  integer- valued 
random  variable 

n 

L„  =  L0  +  y]  8n ,  (3.26) 

k= 0 

for  which  we  assume  that  L0  >  0.  The  increments  of  the  random  walk  process 
{Ln  :  n  >  0}  are  denoted  by  5n  G  {—1,0, 1}  for  transition  epochs  n  G  Z+.  Denote 
the  phase  process  of  the  QBD  to  be  {<pn  :  n  >  0}  and  consider  the  resulting  Markov 
chains 


WL  = 

{(in?  4>n) 

:  n  >  0}, 

Ws  = 

{ 0n) 

:  n  >  0}. 

43 


The  transition  probabilities  of  W&  are  given  by 


P(sn+1  = 

—  1)  0n+l 

=  j  |  Sn,  (pn  =  i ) 

=  (^2  )ij 

(3.27) 

P(sn+1 

=  0,  4>n+l 

=  j  |  Sn,  4>n  =  i ) 

= 

(3.28) 

P(5n+1 

=  1)  4>n  + 1 

=  j\8n,(j)n  =  i) 

=  (^o  )*,,■• 

(3.29) 

We  note  here  that  the  increments  of  the  random  walk  Wl  are  dependent,  as  they 
arise  from  the  transitions  of  the  underlying  QBD,  and  thus  indicate  that  the  process 
embedded  at  level  jumps  is  not  a  Markov  chain.  We  will  deal  with  the  dependency 
issue  later  in  this  discussion;  our  first  task,  however,  is  to  determine  the  steady-state 
behavior  of  the  increment  process  {5n}.  Through  aggregation  of  probabilities,  it  is 
clear  that  the  matrix  A  =  A0  +  Ai  +  A2  contains  the  transition  probabilities  for 
the  phase  process  {4>n}.  Thus,  by  choosing  as  the  initial  probability  vector  for  {4>n} 
the  invariant  probability  vector  ol  corresponding  to  A,  we  initialize  the  process  in 
steady-state,  and  thus,  through  association  to  the  Markov  chain  Ws,  we  may  assume 
that  the  increment  process  {5n}  is  likewise  in  steady-state. 

Define  5  to  be  the  limiting  random  increment  6  =  lim,,-^  8n.  The  process 
Ws,  therefore,  has  the  steady-state  probability  distribution  vector  [aA2,  ctAi,  aA0]. 
The  drift 

d(i)  =  lim  E[8n  |  <5n_i  =  i] 

n— XX) 

associated  to  the  random  walk  Wl  for  all  levels  is  thus  given  by 

d(i)  =  (— 1  )ctA2e  +  0  ■  otAie  +  1  •  aA0e  (3.30) 

=  otA^e  —  a.A2e. 


Since  the  QBD  is  homogeneous  in  its  levels,  its  drift  terms  d(i)  are  independent  of 
level  i,  and  so,  for  the  purpose  of  clarity,  we  define  d  =  d(i),  i  >  1. 
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As  previously  implied,  the  dependence  of  the  increments  Sn  precludes  {Ln} 
from  being  a  Markov  chain.  It  is  therefore  necessary  to  extract  a  suitable  Markovian 
subprocess  of  {Ln}  by  embedding  at  the  appropriate  time  epochs.  As  stated  in  the 
proof  of  [66:  Thm  7.2.3],  these  epochs  may  be  chosen  as  the  times  k0,  ki,  k2, . . .  of 
return  of  the  process  W, 5  to  its  initial  state  (i.e.,  the  first  increment  of  the  random  walk 
{Ln})  Ao-  These  instants  are  renewal  epochs,  a  fact  which  allows  us  to  declare  that 
the  integer- valued  process  {lZn  :  n  >  0}  embedded  at  such  instants  is  a  conventional 
random  walk  with  independent  increments.  Moreover,  its  behavior  is  identical  to 
that  of  the  underlying  QBD  process  until  its  first  return  to  level  0,  after  which 
{lZn}  may  or  may  not  decrement  to  a  negative  level.  We  further  observe  that  the 
increments  of  7 Zn  are  given  by 

kn 

£n  =  &ji 

j=kn-! 

a  fact  that  allows  us  to  compute  the  drift  at  level  i  as  follows: 

de(i)  E!^£n  |  i]  (kn—\  kn)d ,  (3.31) 

the  last  equality  being  a  consequence  of  the  fact  that  the  QBD  is  homogeneous  in 
levels.  It  is  thus  clear  that  d  =  d(i)  <  0  if  and  only  if  de(i)  <  0.  Hence,  by  Pakes’ 
lemma,  it  becomes  apparent  that  d  <  0  is  sufficient  to  guarantee  a  finite  expected 
sojourn  time  from  level  i  back  to  level  i  —  1  for  each  level  i.  Indeed,  as  is  proven 
in  [66:  Thm  7.2.4],  the  invariance  of  drift  over  levels  of  the  QBD  also  makes  this  a 
necessary  condition.  This  enables  ns  to  state  the  following  result: 
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Theorem  3.9.  Suppose  that  we  are  given  an  irreducible,  discrete-time  level- 
independent  QBD  with  a  finite  number  of  phases.  The  QBD  is  positive  recurrent  if 
and  only  if 

olAqE  <  aA2e,  (3.32) 

where  a.  is  the  unique  solution  to  a  A  =  0,  cue  =  1,  and  A  =  A0  +  A\  +  A2. 

We  now  give  a  formal  presentation  of  the  fundamental  period  as  described  in 
[83:  Sec  3.3],  which  has  significant  import  in  any  discussion  of  the  positive  recurrence 
of  a  QBD  and,  as  we  shall  later  see,  a  process  of  M/G/  1-type.  The  concept  of  the 
fundamental  period  is  motivated  by  the  notion  of  the  busy  period  of  a  queue  in  the 
classical  sense  of  the  phrase.  The  busy  period  is  loosely  defined  as  the  first  passage 
time  Tb  from  the  instant  of  the  first  customer  arrival  to  the  queueing  system  until 
the  first  instant  that  the  system  again  becomes  empty,  which,  in  terms  of  a  QBD, 
may  also  be  interpreted  as  the  first  passage  time  from  level  1  back  to  level  0.  For 
the  M/M/1  or  M/G/ 1  queues,  the  busy  period  corresponds  to  a  contiguous  interval 
during  which  the  server  is  processing  work.  However,  this  interpretation  may  not  be 
valid  in  more  complex  models,  particularly  those  in  which  service  may  be  interrupted 
by  failures  or  vacations.  Hence,  we  generalize  the  original  concept  of  a  busy  period 
to  encompass,  not  just  the  first-passage  time  just  mentioned,  but  any  period  that 
begins  with  the  arrival  of  i  +  1  customers  to  the  system  and  ends  with  the  system 
size  decrementing  to  i. 

Using  the  notation  of  [82:  Sec  2.2],  we  begin  its  formal  definition  by  introducing 
the  first-passage  times  T(i  +  r,j ;  i,j ')  from  a  level  i  +  r,  r  >  1  and  Zi+r  =  j  to  level 
i,  i  >  0  and  Z*  =  j',  where  j'  is  the  first  phase  ( hitting  state )  that  the  process 
attains  when  it  reaches  level  i.  We  likewise  define  the  quantities  V(i  +  r,j]i,j') 
of  the  number  of  transitions  that  comprise  a  first-passage  time  T(i  +  r,j]j'),  with 
the  corresponding  value  for  a  fundamental  period  thus  given  by  V{i  +  l,j;j').  We 
then  denote  the  fundamental  period  by  the  stochastic  time  interval  T(i  +  1  ,j',i,f). 
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With  these  definitions  in  hand,  we  may  finally  introduce  the  following  matrix  of 
probabilities 


[G^\ir,x)]jfeS  =  P{T(i  +  r,j]f)  <  x,V{i  +  r,j]f)  =  v}  (3.33) 
and  its  associated  joint  transform  matrix 

oo  /»oo 

Gf{s,z)  =  YV  /  e~axG®(u,x)dx.  (3.34) 

u=0 

Let  &l\v]x)  =  G^\ir,x)]  the  matrix 

OO 

G®(x)  =  ^G(i)(m,x) 

v= 0 

thus  defined  is  called  the  matrix  distribution  of  the  fundamental  period  at  level  i. 
From  the  definitions  given  above,  one  is  able  to  infer  the  relationship  of  the  busy 
period  to  the  fundamental  period,  which  is  that  the  matrix  distribution  of  the  busy 
period  is  given  by  We  also  indicate  the  following  equivalence  (the  matrix 

is  the  level-dependent  version  of  the  matrix  G  defined  in  (3.8)): 

G®  =  GW(0,1),  ieZ+. 

Combining  this  observation  with  Theorem  3.3  formally  establishes  the  connection 
between  the  fundamental  period  and  the  positive  recurrence  of  QBDs. 

If  the  QBD  is  level- independent,  then  the  homogeneity  of  phases  over  all  levels 
guarantees  that  G^(u,x)  =  G^\u,x)  for  any  i,j  G  Z+.  However,  this  is  clearly  not 
the  case  for  a  level-dependent  QBD,  and  so  we  must  therefore  differentiate  between 
the  matrices  G^(u,x)  based  upon  the  starting  level  i.  The  conclusion  that  one  may 
draw  from  this  discussion  is  that  the  distribution  of  the  fundamental  period  varies 
with  the  starting  level  for  level-dependent  M/G/  1-type  processes.  Nevertheless,  the 
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fundamental  period  is  a  class  property,  even  in  the  level-independent  case,  and  thus 
the  conclusions  drawn  in  [82:  Sec  3.3]  for  the  busy  period  likewise  hold  for  the 
level- dependent  case. 

Having  asserted  the  relevance  of  the  level-independent  conclusions,  the  fol¬ 
lowing  theorem  provides  a  means  of  computing  the  distribution  of  a  fundamental 
period  for  a  QBD,  which  is  a  restatement  in  level-dependent  terms  of  Lemma  3.3.2 
and  Theorem  3.3.1  in  [82], 

Theorem  3.10.  The  transform  matrices  G^(z,  s),  z  E  [0, 1],  s  >  0,  are  the  minimal 
nonnegative  solutions  to  the  matrix- quadratic  equations 

X(z,  s )  =  zC^(s)  +  C^(s)X2(z,  s ),  i  e  Z+,  (3.35) 

where 

Cq\s)  =  (si  —  Ai^A^  and  C^^s)  =  (si  —  Afj^A^ . 

Proof.  In  the  proof  of  [83:  Thm  3.3.1],  the  sequence  jGn\;s,,s)  j  ,  n  G  Z+,  is  con¬ 
structed  by  letting  Xq(z,s)  =  0  in  the  recursive  formula 

Xn+l(z,s)  =  zCq\s)  +  C$\s)X2n(z,  s ). 


It  is  subsequently  proved  that  this  nondecreasing  sequence  converges  to  the  limit 
given  by 

lim  G®(z,s)  =  G(i)(z,s). 

n— >00 


□ 


One  may  thus  construct  a  simple  algorithm  based  on  the  following  procedure: 


X0(z,s)  =  0, 

Xn+i(z,s)  =  zC{0l)(s)  +  C^(s)Xl(z,s),  n  >  1.  (3.36) 
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It  is  also  possible  to  obtain  numerical  approximations  of  the  matrices  {G^,  i  > 
0}  via  algorithms  for  the  computation  of  the  steady-state  distribution  of  a  (level- 
dependent)  QBD.  Such  algorithms  usually  produce  the  rate  matrices  R ^  (if  not  the 
G^  matrices  themselves)  as  by-products  of  the  procedure.  One  may  then  employ 
the  level-dependent  version  of  the  relationships  (3.10)  -  (3.12)  in  order  to  obtain 
Gb).  We  shall  discuss  such  algorithms  in  detail  in  Chapter  4. 

3.2  M /G /1-Type  Processes 

In  the  previous  sections,  we  observed  how  the  special  structure  of  the  QBD 
facilitates  the  study  of  a  large  class  of  queues  using  a  relatively  modest  collection  of 
methods.  It  stands  to  reason  that  a  matrix-analytic  analog  to  the  QBD  must  surely 
exist  for  classes  of  non-Markovian  queues  with  highly-structured  embedded  Markov 
chains  as  well.  Indeed,  such  a  class,  epitomized  by  the  classical  M/G/l  queue,  has 
been  defined,  and  a  separate,  but  related  set  of  analytical  tools  to  those  that  apply  to 
QBDs  have  been  discovered  (see  [82]).  In  this  section,  we  review  some  of  the  currently 
known  solution  techniques  for  the  class  of  models  known  as  M/G /1-type  processes , 
both  in  terms  of  computing  a  stability  criterion  and  also  in  the  numeric  computation 
of  the  steady-state  distribution.  Just  as  in  previous  sections  of  this  chapter,  we  draw 
a  distinction  between  level- independent  and  level-dependent  versions  of  this  class  of 
models.  A  specific  method  discovered  in  this  research  for  determining  the  ergodicity 
of  a  level-dependent  M/G/  1-type  process  shall  be  presented  in  Chapter  5,  which 
addresses  the  M/G/l  retrial  queueing  system  modulated  by  a  random  environment. 

3.2.1  Markov  Renewal  Sequences  and  the  M/G/l  Queue 

The  M/G/l  queueing  system,  as  well  as  related  systems  of  M/G/  1-type,  can¬ 
not,  by  virtue  of  its  non-exponential  service  distribution,  be  described  as  a  CTMC, 
nor  does  it  possess  an  embedded  DTMC  at  jump  transitions  in  the  manner  of  its 
Markovian  counterpart.  In  order  to  determine  the  appropriate  embedded  DTMC 
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(i.e.  to  facilitate  analysis  of  the  non-Markovian  system),  we  resort  to  the  theory 
of  Markov  renewal-processes.  The  idea  is  to  construct  a  random  bivariate  sequence 
{(Yn,Sn)  :  n  >  0},  or  Markov  renewal  sequence  (MRS),  at  suitable  epochs  Sn  in 
the  system-size  process  (A"(f)  :  t  >  0}  of  the  M/G/l  queue  in  such  a  way  that 
{Yn  :  n  >  0}  is  a  Markov  chain,  where  Yn  =  X(S+).  We  may  then  study  the 
steady-state  distribution  of  the  embedded  Markov  chain  {Yn}  in  order  to  study  the 
recurrence  properties  of  the  continous-time  system. 

We  will  not  give  the  full  details  of  the  definition  of  a  MRS  here;  for  this, 
we  refer  the  reader  to  [61:  p  479].  The  key  characteristic  is  that  the  successive 
transitions  of  {(Yn,  Sk)}  should  be  independent  of  its  history,  or,  in  other  words,  the 
Markov  property  should  hold  for  the  MRS.  We  next  let  rn  =  Sn+i  —  Sn  and  define 
the  probabilities 


Q*j(x)  =  P  {Yn+ 1  =  j,  rn  <  x  |  Yn  =  i}  i,  j  e  S. 

The  matrix  Q*(x)  =  [Q*j(x)  is  called  the  kernel  of  the  MRS.  The  discrete  random 
variable  N(t),  which  is  the  number  of  observations  of  the  MRS  up  to  time  t,  defines 
a  Markov  renewal  process  { N(t )  :  t  >  0}.  If  N(t)  records  the  number  of  transitions 
up  to  time  t,  then  the  process 


(A"(f)  =  YN{t)  :  t  >  0} 


forms  a  semi-Markov  process  (SMP). 

It  should  be  emphasized  that  the  sequence  {(Yn,Tn)  :  n  >  0}  is  also  often 
referred  to  as  the  MRS;  nevertheless,  the  two  definitions  clearly  uphold  the  same 
idea  with  regard  to  the  semi-Markov  process  that  they  commonly  define.  We  now 
focus  on  the  utility  of  the  MRS,  which  is  embodied  in  the  statement  of  the  following 
theorem  which  appears  in  [61:  Thm  9.1]  and  is  significant  to  the  determination  of 
the  stability  condition  for  the  non-Markovian  system: 
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Theorem  3.11.  If  {{Yn  ,Sn)  :  n  >  0}  is  a  MRS,  then  the  process  {Yn  :  n  >  0}  is  a 
DTMC  with  transition  probability  matrix  given  by  Q*(oo). 

We  therefore  wish  to  obtain  the  matrix  P  =  Q*(oo )  and  explicitly  define  those 
conditions  that  guarantee  the  ergodicity  of  the  embedded  Markov  chain,  and  thus 
of  the  original  queueing  system. 

Just  as  in  Neuts  [82],  we  use  the  standard  M/G/l  queue  with  arrival  rate  A  and 
with  service  distribution  H(-)  with  average  1/p.  as  a  simple  example  of  a  queueing 
system  whose  embedded  semi-Markov  kernel  demonstrates  the  aforementioned  struc¬ 
ture.  Using  the  definitions  given  for  Sn,  Yni  and  rn  given  in  the  previous  paragraphs, 
we  will  consider  the  Markov  renewal  process  embedded  at  instants  Sn  just  after  ser¬ 
vice  completion  and  defined  by  the  Markov  renewal  sequence  {( Yn ,  rn)  :  n  >  0}.  For 
i,i'  >  0  and  for  x  >  0,  standard  renewal  arguments  are  used  to  obtain 


f*  Xe  XuQl  it(x  —  u)  du,  i  —  0,  i!  >  0, 

QUx)  =  I  /c 
0, 


(A  uY~i+l 


ro  e  Xu  dH(u),  i  >  1,  if  >  i  -  1, 

u  \v  —  i  +  1)! 


i'  <  i  —  1. 


Define  the  following  joint  probabilities  for  x  >  0  and  v  —  0, 1,  2, . . .: 


Bu(x)  =  P  {v  arrivals  occur  during  (0,  r],  r  <  x,  \  Yq  =  0}  , 
Av{x)  =  P  {v  arrivals  occur  during  (0,  r],  r  <  x,  \  Y0  >  0}  . 


It  is  evident  from  these  definitions  that 


Bi>(x) 


Ai'-i+ i(x) 


Qw(x),  i’>  0 
QUx),  i  >  1,  i'  >  0. 
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If  we  let  x  tend  towards  infinity  and  define 


Bu  =  lim  Bv(x),  Av  =  lim  Av(x), 

>00  X—>00 


we  obtain  the  transition  probability  matrix  P  =  Q*(oo)  of  the  embedded  Markov 
chain,  which  assumes  the  following  structure: 


Bq  B  i  B2  B3  Ba 

Aq  Ai  A2  A3  A4 

0  A0  Ai  A2  A3 

0  0  A0  A\  A2 

0  0  0  Aq  Ai 


(3.37) 


As  we  shall  soon  see,  processes  with  embedded  transition  probability  matrices  of 
the  canonical  form  (3.37)  comprise  a  distinct  class  of  stochastic  processes  with  a 
common  set  of  steady-state  solution  methods. 

3.2.2  Level-Independent  Processes 

Stochastic  processes  whose  embedded  Markov  chains  exhibit  the  characteristic 
form  (3.37)  are  termed  level-independent  processes  of  M/G/l  type.  This  is  the  sim¬ 
plest  manifestation  of  the  M/G/  1-type  process  in  that  the  terms  Ai,  i  =  1,2,3, .. . 
do  not  vary  based  upon  row  membership.  As  the  form  of  this  matrix  shows,  the 
embedded  Markov  chain  may  transition  across  any  number  of  columns  to  the  right 
while  being  restricted  to  a  single  transition  to  the  left.  This  is  the  reason  why 
M/G/  1-type  processes  are  often  referred  to  as  skip-free-to-the-left.  Although  for  the 
general  M/G /1-type  Markov  chain  it  is  possible  to  specify  infinite-dimensional  block 
terms  Av  and  Bv  for  v  >  0,  we  shall  assume  henceforth  that  the  blocks  are  finite  with 
dimension  m.  Also,  as  with  the  definition  of  the  QBD  transition  probability  matrix, 
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we  denote  the  first  coordinate  of  each  state  as  the  level  and  those  that  comprise  the 
remainder,  phases. 

For  the  sake  of  completeness,  we  present  here  a  dual  class  of  processes  of 
GI/M /1-type  for  which  P  takes  the  canonical  form 


Co 

Do 

0 

0 

0 

Ci 

D\ 

Do 

0 

0 

c2 

d2 

Dl 

Do 

0 

c3 

d3 

d2 

Dl 

D{ 

c4 

d4 

d3 

D2 

D 

(3.38) 


As  in  the  case  of  processes  of  M/G/  1-type,  the  structure  of  this  matrix  makes  clear 
the  origin  of  the  term  skip-free-to-the-right.  The  analysis  of  such  systems  is  consid¬ 
erably  easier  than  that  of  type  M/G/ 1  due  to  the  fact  that  the  matrix- geometric 
property  holds  for  the  steady-state  probabilities  of  system  size,  ft  is  interesting  to 
note  that  QBDs  are  skip-free-to-the-right  and  -to-the-left  simultaneously,  and  are 
consequently  of  both  GI/M/ 1-  and  M/G/  1-type.  In  particular,  the  classification  of 
QBDs  as  GI/M/  1-type  processes  is  consistent  with  the  matrix-geometric  nature  of 
their  steady-state  probabilities. 

The  (steady-state)  theory  of  M/G/  1-type  processes  has  been  well-established 
for  quite  some  time  (see  [82])  and  applies  to  a  large  number  of  related  systems.  The 
next  class  of  processes  that  we  discuss  does  not  possess,  at  least  to  the  same  degree,  a 
generally-defined  set  of  methods  that  allow  for  its  easy  solution.  In  what  follows,  we 
will  review  the  properties  of  level-dependent  M/G/  1-type  processes  that  are  relevant 
to  the  determination  of  their  stability  and  to  the  computation  of  their  steady-state 
distributions.  This  will  later  be  used  in  the  formulation  of  a  new  condition  for  the 
stability  of  such  systems  in  Chapter  5. 
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3.2.3  Lev  el- Dependent  Processes 

A  natural  generalization  of  the  concept  of  M/G/  1-type  processes  is  that  in 
which  the  embedded  transition  probability  matrix  [Pw]  is  dependent  upon  i,  which 
is  the  previous  number  in  system.  In  this  case,  we  obtain  the  following  form  of  the 
transition  matrix: 


B0 

B1 

»2 

B 3 

B 4  ••• 

d(1) 

A? 

4° 

41' 

41}  ... 

0 

d(2) 

42> 

42) 

42)  ••• 

0 

0 

,(3) 

43) 

43)  ••• 

0 

0 

0 

4(4) 

44)  ... 

(3.39) 


in  which  the  distributions  of  the  increments  of  the  process  vary  depending  upon  the 
current  number  in  system.  We  denote  processes  with  embedded  transition  proba¬ 
bility  matrices  of  the  form  (3.39)  level-dependent.  Otherwise,  we  shall  refer  to  the 
process  as  being  level-independent.  As  the  reader  may  recall,  an  analogous  distinction 
was  made  in  the  case  of  QBDs,  and  which  required  a  bit  of  effort  in  order  to  char¬ 
acterize  the  stability  conditions  that  prevailed  in  the  more-general  level-dependent 
type  of  process. 

3.2.4  Ergodicity  Conditions 

As  hinted  previously,  there  exists,  for  an  embedded  level-independent  Markov 
chain  of  M/G/  1-type,  a  well-developed  theory  of  ergodicity,  as  evidenced  by  the 
many  publications  on  the  subject;  see  [8,  12,  13,  22,  67,  82,  89,  92],  among  others. 
What  follows  here  is  a  review  of  basic,  but  essential  facts  concerning  the  stability  and 
steady-state  distribution  of  irreducible  processes  of  M/G/l-type  as  it  is  presented 
in  [82:  Chap  2],  We  first  consider  the  criteria  that  determine  the  ergodicity  of  the 
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embedded  Markov  chain  Q*( oo).  As  noted  by  Neuts  [82]  and  others,  it  is  important 
to  recognize  the  fact  that  the  ergodicity  of  the  embedded  chain  is,  in  general,  only 
a  necessary  condition  for  the  recurrence  of  the  associated  Markov  renewal  process 
Q*(-),  as  one  must  further  stipulate  that  the  mean  increment  of  the  MRP  for  any 
given  time  interval  x  >  0  is  finite.  However,  as  in  the  case  of  the  QBD,  the  ergodicity 
of  the  embedded  chain  is  also  sufficient  if  the  process  has  a  finite  number  of  phases; 
e.g.,  a  process  modulated  by  a  random  environment  with  a  finite  state  space.  Since 
we  assume  such  a  process  in  this  dissertation,  it  becomes  necessary  only  to  ascertain 
the  conditions  that  guarantee  the  ergodicity  of  the  embedded  chain. 

Central  to  this  concept  of  the  ergodicity  of  an  M/G/  1-type  Markov  chain  is  the 
fundamental  period ,  which  was  defined  in  Section  3.1.3  for  the  QBD.  This  quantity 
is  of  paramount  importance  to  the  determination  of  the  recurrence  properties  of  the 
states  of  any  queueing  system.  Using  our  previous  notation,  let  us  now  consider  the 
probabilities  G(v,x)  =  G \{y,x).  We  then  have  the  following  relationship  between 
these  matrices  of  mass-functions  and  the  matrices  Av(x)  of  transition  probabilities, 
a  result  that  appears  in  [82:  p  80]: 

Theorem  3.12.  The  matrices  G(u,x)  satisfy  the  difference  equations 

OO 

G(l,x)  =  A0(x),  G(u,x)  =  Ak(x)  *  Gk(u  -  l,x),  v>2,  (3.40) 

k= 1 

where  (•,  •,*)  is  the  matrix  convolution  product  defined  in  Appendix  A,  (1.4). 

The  preceding  theorem  makes  rigorous  the  notion  of  the  fundamental  period  as 
potentially  including  one  or  more  periods  between  transition  epochs  of  the  embedded 
Markov  chain. 

We  next  define  the  familiar  matrix  G(x)  of  probability  mass  functions  as 

OO 

G(x)  =  G(u,  x),  x  >  0,  (3-41) 

v=Q 
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which  thus  satisfies  the  relation  G(x)e  <  e.  The  matrix  G(x )  again  denotes  the 
distribution  matrix  of  the  fwidamental  period.  As  such,  we  may  interpret  G  =  G{ oo) 
as  the  matrix  whose  (j,  j^-th  entry  is  the  probability  that  the  process  will  attain  the 
state  ( i,j '),  given  that  it  started  in  ( i  +  1,  j).  Also,  as  shown  in  [82:  Sec  2.3],  G  is 
the  minimal  nonnegative  solution  to  the  equation 

OO 

g  =  (3.42) 

v=0 

The  following  Lemma  is  clear  from  the  probabilistic  interpretation  of  G  and  the 
definition  of  recurrence: 

Lemma  3.1.  If  the  M /G /1-type  process  with  the  embedded  Markov  chain  whose 
transition  probability  matrix  is  given  by  Q*  =  Q*( oo)  is  recurrent,  then  the  matrix 
G  =  G( oo)  is  stochastic. 

Thus,  it  is  necessary  that  G  be  stochastic  in  order  for  the  process  be  recurrent. 
Although  we  ultimately  seek  to  define  the  necessary  and  sufficient  criteria  for  positive 
recurrence  to  hold,  it  is  first  necessary  to  define  an  attainable  analytic  condition  that 
is  equivalent  to  the  stochasticity  of  the  matrix  G. 

It  is  generally  known  from  Perron-Frobenius  theory  that  a  given  square  ma¬ 
trix  M  is  stochastic  if  and  only  if  its  maximum  positive  eigenvalue  (i.e.,  its  Perron 
eigenvalue)  sp(M )  is  equal  to  unity.  A  simple  analytical  condition  that  guarantees 
the  fulfillment  of  sp(M )  =  1  has  been  derived  in  [82:  Chap  3].  In  order  to  state  this 
condition,  we  first  define  the  matrix  ^-transform 

OO 

A*(z)  =  Y,A"zV 

u=0 

and 

A  00 

P  =  ~rA*(z)  =  X/ uAv' 

az  z=l-  v= 0 
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where  \z\  <  1.  Suppose  as  well  that  7r  is  the  invariant  probability  vector  associated 
to  the  stochastic  matrix  A  =  71*  (1“).  We  may  now  state  the  following  theorem  (see 
[82:  Thm  2.3.1]): 

Theorem  3.13.  If  the  matrix  A  is  irreducible,  then  the  matrix  G  is  stochastic  if  and 
only  if 

p  =  n/3e  <  1.  (3.43) 

Hence,  the  embedded  Markov  process  Q*  =  Q*(o o)  is  recurrent  if  and  only  if  inequal¬ 
ity  (3.43)  holds. 

One  must  keep  in  mind,  however,  that  Theorem  3.13  only  guarantees  recurrence 
of  the  embedded  Markov  chain.  Positive  recurrence  of  a  discrete-time  Markov  chain 
{Xn  :  n  >  0}  requires  that  for  any  initial  state  i,  fj  =  E[Tj  \  X0  —  i]  <  oo,  where  Tj  is 
the  first  time  to  reach  the  state  j,  and  i  ^  j.  Since  the  transition  probability  matrix 
of  the  embedded  Markov  chain  Q*( oo)  is  irreducible,  the  recurrence  properties  of 
every  state  is  exactly  the  same,  and  so  it  suffices  to  verify  this  condition  for  the 
initial  state  0.  One  may  thus  conclude  that  positive  recurrence  holds  if  and  only  if 

OO 

^  uBv  <  oo. 

It  is  also  shown  in  Section  3.2  of  [82]  that  p  =  1  corresponds  to  the  null-recurrent 
case.  Taken  together,  these  facts  lead  us  to  the  following  conclusion: 

Theorem  3.14.  Suppose  that  the  matrix  G  that  corresponds  to  the  embedded  Markov 
process  Q*(oo)  is  irreducible.  Then  the  process  Q*(oc)  is  positive  recurrent  if  and 
only  if  p  <  1  and  Y^Lo  <  oo. 

Since,  by  definition,  the  matrix  f3  contains  the  conditional  expectations  of 
the  number  of  arrivals  during  a  fundamental  period  of  the  embedded  Markov  chain 
<5*(oo),  p  may  be  interpreted  as  the  expected  number  of  arrivals  during  a  funda¬ 
mental  period.  It  is  intuitive  that  if  this  number  exceeds  unity,  then  the  queue  will 
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tend  to  grow  in  the  long  run.  This  interpretation  is  essential  to  the  formulation  of 
the  stability  criteria  for  level-dependent  M jGj  1-type  queues. 

This  chapter  has  reviewed  the  fundamental  concepts  and  definitions  required 
for  the  steady-state  analysis  of  quasi-birth-and-death  and  M/G/  1-type  systems.  The 
material  presented  here  is  crucial  to  understanding  the  main  results  obtained  in  this 
dissertation  which  are  presented  in  the  next  two  chapters.  The  next  topic  is  the 
formulation  and  steady-state  analysis  of  the  unreliable  M/M/1  retrial  queue  in  a 
random  environment,  for  which  the  material  in  Section  3.1  shall  be  relevant.  Once 
the  results  for  the  Markovian  model  have  been  established,  then  it  will  be  shown 
that  a  similar  queueing  model  with  generally  distributed  service  requirements  is  a 
process  of  M/G/  1-type.  This  will  subsequently  allow  the  use  of  the  theory  and 
methods  described  in  Section  3.2.  Results  for  system  stability  have  likewise  been 
extended  from  the  level-independent  results  of  this  chapter  and  will  appear  in  the 
subsequent  chapters. 


4.  Exponential  Service  Requirements 

This  chapter  provides  the  mathematical  model  description  and  main  results  for  a 
single-server  unreliable  retrial  queue  in  a  random  environment  with  exponential 
services  and  Poisson  arrival  process.  We  first  characterize  the  system  as  a  quasi¬ 
birth- and- death  (QBD)  process,  which  facilitates  a  matrix- analytic  approach.  Once 
this  has  been  accomplished,  the  theory  of  quasi-birth-and-death  processes  may  be 
brought  to  bear  on  the  tasks  of  (1)  determining  the  conditions  under  which  a  steady- 
state  distribution  of  orbit  size  exists,  and  (2)  the  computation  of  the  approximate 
steady-state  distribution  and  related  performance  measures. 

4-1  Model  Description  and  Notation 

Consider  a  single-server  M/M/1  retrial  queue  in  which  the  model  parameters 
(arrival,  service,  failure,  repair,  and  retrial  rates)  are  all  modulated  by  an  external 
random  environment  (see  Figure  4.1).  The  random  environment  is  assumed  to  be 
an  irreducible  continuous-time  Markov  chain  (CTMC)  with  a  finite  state  space  S  = 
{l,...,m},  infinitesimal  generator  Q  =  [fc]ije{i,...,m}  and  stationary  probability 
vector  p  =  [pi, . . .  ,pm\.  When  the  environment  is  in  state  j ,  customers  arrive  to 
the  system  according  to  a  Poisson  process  with  rate  A  j  >  0  while  their  service 
requirements  are  exponentially  distributed  with  rate  pj  >  0.  When  the  server  is 
either  idle  or  busy,  breakdowns  occur  according  to  a  Poisson  process  with  rate  £,•  > 
0,  and  the  subsequent  exponential  repair  time  has  rate  oq  >  0.  Define  the  Tri¬ 
dimensional  vectors  A  =  (Ai,  A2, . . . ,  Am),  p  =  (pi,  p>, ....  pm).  £  =  (fi,  f2,  •  •  • ,  fm), 
and  ex.  =  (cui,  a2, . . . ,  am).  Retrial  customers  attempt  to  regain  access  to  the  server 
independently  of  all  other  customers  at  exponentially  distributed  time  intervals  with 
rate  9j  >  0  when  the  environment  is  in  state  j.  The  vector  of  retrial  rates  is 
6  =  {9 1,  #2,  •  •  • ,  9m).  For  a  row  vector  x,  we  define  the  diagonal  matrix  of  its  elements 
by  A  (a?)  =  diag(x).  Arrival,  service,  failure,  repair,  and  retrial  processes  are  assumed 
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Random  Environment 


to  be  mutually  independent.  Denote  by  R(t),  the  number  of  customers  in  orbit  at 
time  t  and  let  Z(t)  be  the  state  of  the  random  environment  at  time  t.  The  random 
variable  X(t)  denotes  the  status  of  the  server  defined  by 


X{t)  =  { 


0 

1 

2 


if  the  server  is  failed  at  time  t 

if  the  server  is  operational  but  idle  at  time  t 

if  the  server  is  operational  and  busy  at  time  t 


The  continuous-time  stochastic  process,  {(R(t) ,  Z (t) ,  X (t))  :  t  >  0},  has  state 
space  S  =  {  (i,j,  k)  :  i  e  Z+,  j  e  S,k  G  {0, 1, 2}  },  where  Z+  is  the  set  of  nonnegative 
integers.  It  is  clear  from  the  foregoing  definitions  that  the  process  is  a  multivariate 
Markov  chain  with  one  infinite  dimension,  namely  the  number  of  customers  in  orbit. 
Next,  we  show  that  {(i?(t),  Z(t),X(t))  :  t  >  0}  can  be  viewed  as  a  tri-layered  QBD 
and  exploit  this  structure  to  obtain  the  stability  condition  for  the  queueing  sys¬ 
tem  and  to  obtain  the  approximate  steady-state  distribution  and  mean  performance 
measures. 
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Let  us  denote  the  infinitesimal  generator  of  {(R(t) ,  Z (t) ,  X (t))  :  t  >  0}  by  Q*. 
Figure  4.2  shows  scalar  elements  of  the  infinitesimal  generator  are  arranged  in  m  x  m 
diagonal  sub-blocks  corresponding  to  the  m  states  of  the  random  environment.  In 
this  figure,  we  are  restricted  to  displaying  the  entries  of  Q*  when  m  =  3.  Using 
the  A-terminology  to  denote  diagonal  matrices,  we  aggregate  terms  and  rewrite  the 
level- 1  generator  in  block  form  as  follows: 


Q 


* 

l 


C0  A(£)  A(A)  0  0  0  0 

A(a)  D\  0  0  A(A)  0  0 

A(fi)  0  D2  0  A(£)  A(A)  0 

0  0  A  (61)  Cl  A(£)  A(A)  0 

0  0  0  A(a)  D\  0  0 

0  0  0  A(fjt)  0  D2  0 


(4.1) 


where  the  matrices  D\ ,  D2,  and  Cj,  i  >  1  are  given  by 

D\  =  Q  —  A(A  +  a) , 

D2  =  Q  —  A(A  +  /it  +  ^) , 

Ci  =  A(/i)  +  D2  -  iA(0) ,  i  —  1,2, — 
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Visual  inspection  of  (4.1)  does  not  help  us  discern  if  Q*  is  a  QBD.  How¬ 
ever,  we  observe  that  the  diagonal  blocks  repeat  in  cycles  of  length  three,  namely 
{Ci,  Di,  D2}  as  i  — >  oo.  This  suggests  a  higher-order  aggregation  of  3  x  3  block 
entries  (or  3m  x  3m  scalar  entries),  from  which  we  obtain  the  top-level  (0)  matrix 
form  for  the  infinitesimal  generator  Q *: 

T0  A  0  0  0 

0i  T1  A  0  0 

0  ©2  T2  A  0 

0  0  ©3  T3  A 

o  o  o  ©4  r4 

where,  for  i  >  0, 

Q  A(0  A(A) 

T*  =  A(a)  Dl  0  ,  (4.3) 

A(/x)  0  D2 

0  *A(0) 

0  0  ,  (4.4) 

0  0 

0  0  0 

A  =  0  A(A)  0  •  (4.5) 

0  A(0  A(A) 

At  this  level  of  grouping,  it  is  clear  that  Q*  possesses  the  tri diagonal  form  of  a  QBD, 
and  thus,  the  following  theorem  may  be  stated: 
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Theorem  4.1.  The  process  {(R(t),  Z(t),X(t))  :  t  >  0}  is  a  tri-layered,  level- 
dependent  QBD. 

As  of  the  writing  of  this  dissertation,  known  results  for  multi-layered  and  level- 
dependent  QBDs  do  not  provide  much  guidance  in  determining  stability  conditions 
for  the  retrial  queueing  model  {(R(t) ,  Z (t) ,  X (t))} .  Consequently,  such  theoreti¬ 
cal  conditions  must  be  established  before  an  attempt  to  quantify  the  performance 
measures  of  the  system  can  be  made.  In  the  next  section,  we  give  conditions  for  the 
stability  of  general  level-dependent,  discrete-time  QBDs.  What  is  more,  we  will  show 
that  these  conditions  depend  upon  the  QBD  structure,  and  not  upon  the  number  of 
layers  that  it  encompasses. 

4-2  Stability  and  Steady- State  Analysis 

We  now  set  out  to  prescribe  the  analytic  conditions  to  guarantee  the  ergod- 
icity  of  the  unreliable  retrial  queueing  system  in  a  random  environment.  In  what 
follows,  we  present  the  formulation  and  subsequent  proof  of  the  conditions  for  the 
ergodicity  of  general  discrete-time  level-dependent  QBDs.  The  finite  cardinality  of 
the  phase  spaces  (namely  the  environment  and  server  states)  corresponding  to  each 
level  i  allows  the  use  of  the  discrete  embedded  chain  at  transitions  of  the  original 
continuous-time  QBD  process  in  determining  conditions  on  the  model  parameters 
that  guarantee  its  stability.  An  explicit  traffic  intensity  formula  p,  which  does  not 
depend  upon  the  initial  level  i,  will  then  be  derived  for  the  M/Ad/1  model  described 
in  this  chapter. 

As  its  name  suggests,  a  (discrete-time)  level-dependent  QBD  possesses  drift 
values  that  vary  with  the  level  of  the  process.  This  introduces  a  major  difficulty  in 
that  the  nonstationary  nature  of  the  increments  of  the  random  walk  Cn  =  L^n  ren¬ 
der  the  epochs  ki,  k2,  k3, . . .  of  the  return  of  the  process  {5n}  to  h0  non-regenerative. 
Nevertheless,  we  shall  show  that  there  indeed  exists  a  criterion  based  upon  the  level- 
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dependent  drift  d(i)  that  enables  the  determination  of  the  stability  of  the  QBD. 
Before  presenting  this  criterion,  however,  we  will  re-introduce  the  random-walk  pro¬ 
cess  and  associated  terminology  as  they  pertain  to  a  general  level-dependent  QBD. 

We  begin  as  before  with  an  analysis  of  the  random  walk  Ln  =  L0  +  ]T^=1  Sf.  at 
steady-state,  where  each  term  is  defined  as  in  (3.26).  The  definition  of  the  processes 
Wl  and  W$  and  the  corresponding  limiting  increment  5  are  identical.  This  time, 
however,  the  drift  d(i)  at  each  level  i  >  1,  where  d[i)  =  E[8n\  Ln_i  =  i],  varies 
according  to  the  current  level  i.  We  may  thus  consider  d(i)  to  be  the  long-run  drift 
of  the  process  restricted  to  level  i,  i  =  0,1,2,....  Contrast  this  definition  to  the 
analogous  one  for  level-independent  QBDs,  whose  homogeneity  of  drift  over  levels 
allows  us  to  state  the  correspondence  d(i)  =  d(j)  =  d  for  all  i,  j  >  0. 

For  a  discrete-time,  level-independent  QBD  with  a  finite  number  of  phases,  we 
have  seen  that  the  related  Markov  chain  Wl  has  the  finite  steady-state  distribution 


(ad2e,  aAxe,  aA0e ), 


but  its  level-dependent  counterpart 

(a^A^e,  a^A^e,  a^A^e),  i  =  0, 1, 2, ... , 

is  now  infinite-dimensional.  Nevertheless,  just  as  in  the  level-independent  case,  we 
may  evaluate  the  drift  associated  with  level  i,  d(i)  as 

d(i)  =  a^A^e  —  a^A^e,  i  =  0, 1, 2, ... ,  (4.6) 

with  being  the  minimal  positive  solution  to  the  system  of  equations  given  by 

a(i)A®  =  a(i),  a(i)e  =  1,  i  =  0, 1,  2, . . . 
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and  A^  =  is  the  transition  probability  matrix  of  the  QBD  restricted 

to  level  i.  Note  that,  by  virtue  of  the  dependence  of  the  drift  upon  level  index  i,  that 
Theorem  3.9  does  not  apply  to  the  level-dependent  case.  Despite  this,  we  may  still 
establish  a  single  condition  for  ergodicity  based  upon  the  convergence  of  the  sequence 
{dfi)}  as  i  grows  without  bound.  The  sufficiency  of  this  condition  is  provided  by 
Pakes’  Lemma.  However,  the  much  more  difficult  task  of  demonstrating  necessity 
shall  require  the  use  of  Theorem  3.7,  after  which  we  shall  state  the  corresponding 
result  for  continuous-time,  level-dependent  QBDs. 

It  is  now  apparent  that  we  require  a  suitable  embedded  Markov  chain.  By 
observing  that  the  sequence  {(£n,»S'n)  :  n  >  0},  which  is  embedded  at  level-jump 
epochs  Sn  such  that  £n  =  Xf  ,  is  a  Markov  renewal  sequence,  we  show  that  {£n}  is 
a  Markov  chain. 

Lemma  4.1.  Let  {( Xn,Yn )  :  n  >  0}  be  a  discrete-time,  level-dependent  QBD,  and 
let  Sn  denote  the  time  of  the  nth  level  jump  with  S0  =  0.  Define  £n  =  Xf  ,  the  level 
of  the  QBD  just  after  Sn.  Then  {£n  :  n  >  0}  is  a  Markov  chain. 

Proof.  We  shall  first  demonstrate  that  the  increments  of  £  are  PH-distributed  (see 
Section  A. 2  of  the  Appendix).  From  previous  discussions,  it  is  clear  that  the  matrix 
A'\  \  i  >  0,  contains  the  probabilities  for  transitions  within  a  given  level  i,  while 
Aq  +  A^  contains  those  for  transitions  out  of  level  i  conditioned  upon  any  of  the 
states  Sg'1  =  {1,  2, . . . ,  M}  in  the  phase  process  {0^  :  n  >  0}  for  level  i.  We  may 
thus  represent  the  time  the  process  £  spends  between  transitions  as  the  time  until 
absorption,  where  the  absorbing  state  shall  be  denoted  by  0  and  the  conditional 
probabilities  of  absorption  are  given  by  the  vector  (A^  +  A^Je.  Thus,  the  inter¬ 
transition  times  are  PH-distributed  with  the  representation  [7^,  A^],  where  7^  is 
an  initial  probability  vector. 

As  a  consequence  of  the  PH-distributions  of  increments  £i+i  —  £t  according  to 
parameters  that  reside  only  within  a  particular  level  i ,  the  Markov  property  holds 
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over  such  increments,  and  thus 


{(£„,  Sn)  :  n  >  0} 


is  a  Markov  renewal  sequence.  Thus,  by  Theorem  3.11,  we  have  that  {£„}  is  a 
Markov  chain.  □ 


Finally,  we  define  certain  quantities  that  are  needed  for  the  proof  of  our  main 
theorem.  The  first  is  simply  a  restatement  of  the  drift  terms  d(i)  as  ratios,  or 
expressions  of  traffic  intensity,  see  [83]  for  a  detailed  discussion  of  the  traffic  intensity 


*  =  0,1,2,.... 


(4.7) 


In  order  to  make  this  expression  well-defined,  we  will  exclude  the  case  in  which  one 
or  more  members  of  the  sequence  {A^}  has  a  zero  eigenvector.  It  is  easy  to  see  that 
(4.7)  and  the  drift  at  level  i  are  related  by  the  expression 


d(i)  =  (p[i)  -  1  )(aw4'}e)  i  =  0, 1, 2, ... .  (4.8) 

In  order  to  simplify  certain  expressions  that  contain  these  terms,  we  define  7^  = 
oc^Aj^e,  i,j  =  0,1,2,  which  will  allow  us  to  then  rewrite  (4.8)  in  terms  of  the 
traffic  intensity  p<l>  as 


d{i)  =  (pw-  1)7?,  *  =  0,1,2,....  (4.9) 

We  may  now  state  and  prove  the  following  theorem. 

Theorem  4.2.  An  irreducible,  aperiodic,  discrete- time,  level  dependent  QBD  is  er- 
godic  if  and  only  if 

p  =  lim  p(A  <  1,  (4.10) 

i^oo 

with  p^A  defined  as  in  (4.7). 
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Proof.  We  first  assume  that  inequality  (4.10)  holds.  By  (4.9)  we  obtain 


lim  sup  d{i)  =  (p  —  1)  lim  sup  7^.  (4.11) 

i— » 00  i— xx 

Assume  that  p  <  1.  Since  it  is  impossible  to  have  an  increment  (a  jump  between 
levels)  of  magnitude  greater  than  unity  in  a  QBD,  the  finite-increment  stipulation  of 
Pakes’  Lemma  (c.f.  [87]  and  [61:  pp  96-97])  is  fulfilled.  Next,  we  observe  that  since 
72 ^  >  0  for  all  i,  we  likewise  obtain  that  lim  snp,^^  7^  >  0.  Consequently, 

lim  snp  d{i)  =  (p  —  1)  lim  sup  7?  <  0- 

2— XX)  i—>  00 

With  the  remaining  condition  of  Pakes’  Lemma  satisfied,  we  may  conclude  that  the 
QBD  is  positive  recurrent  if  p  <  1. 

The  proof  of  necessity  hinges  upon  the  fulfillment  by  the  QBD  of  Kaplan’s 
condition  (see  Section  3.1.3).  Let  z  G  [0,1)  and  define  the  functions 

^i{z)  =  i^z1 /(l- z),  i>0  (4.12) 

as  in  (3.25).  It  was  shown  in  Section  3.1.3  that  the  equivalence  d(i)  =  lirrq^]-  ipi(z) 
holds,  and  thus  demonstrates  the  relationship  of  the  function  fjfz)  to  the  drift 
condition  of  the  Markov  chain.  It  is  also  mentioned  that  the  fulfillment  of  Kaplan’s 
condition  implies  that  the  drift  quantities  d(i),i  >  0  are  bounded  below.  If,  in 
addition,  we  have  that  d(i)  >  0  for  every  %  past  a  certain  level  N ,  then  it  becomes 
clear  from  this  discussion  that  the  Markov  chain  cannot  be  ergodic;  this  is  the 
reasoning  behind  Theorem  1  of  [95] ,  which  we  shall  show  holds  under  the  hypothesis 
of  our  theorem.  In  doing  so,  we  thus  prove  the  contrapositive  of  the  statement  of 
necessity,  which  will  complete  our  proof. 

Assume  that  p  >  1  and  that  the  system  is  stable;  we  need  to  show  that  the 
QBD  cannot  be  ergodic.  Using  the  same  reasoning  as  in  the  proof  of  sufficiency,  we 


assert  that  d(i)  <  oo  for  each  i  >  0.  In  order  to  prove  that  Kaplan’s  condition  holds, 
we  will  derive  the  function 

Mz)  =  (^zl  ~YlPiiz^j  /(1  “  z)i  i  -  °>  2  G  [°>  !) 

specifically  for  the  Markov  chain  {(5„,  <pn)  :  n  >  0}  consisting  of  the  increments  of  the 
random  walk  Ln  (3.26).  Thus,  Sn  may  assume  any  one  of  the  three  values  {  —  1,  0,  +1} 
with  the  corresponding  steady-state  probabilities  •  •  •]>  where  7 b)  = 

[72^,  7i*\  7o^]-  This  results  in  the  following  expression: 

fpi(z)  =  [t?  +  (t?}  ~  X) 2  +  7?}-2]5  *  >  !•  (4T3) 


We  next  investigate  the  behavior  of  the  sequence  of  terms  {'tpi(zi)  :  i  —  2,  3, . . .} 
evaluated  at  the  critical  points  zt  for  which  ^  achieves  a  local  minimum  over  the 
interval  (0, 1).  After  some  tedious  algebra,  we  obtain 


(4.14) 


and  thus, 

A(zi)  =  “  7)  72°  1  -  •  (4-!5) 

Because  the  probability  of  downward  drift  increases  with  the  level,  it  is  seen  that 

{72^}  is  nondecreasing,  and  thus,  so  is  {^ (£*)}.  Consequently,  it  may  be  readily 

verified  that  ipi(zi)  >  0  for  i  >  2.  We  observe  that,  for  each  i  >  2,  'ipi(z)  is  non¬ 

decreasing  in  z  on  the  interval  [2i,  1).  This,  together  with  the  continuity  of  over 
the  interval  [0, 1)  further  implies  that,  for  each  i  >  2,  ipi(z)  >  0  for  all  z  G  [ 7, 1). 
Finally,  we  observe  that  {1  /p^}  is  a  nondecreasing  sequence,  which  is  also  due  to 
the  fact  that  drift  decreases  as  the  level  increases  due  to  the  geometric  rate  of  decay 
of  the  steady-state  probabilities  in  an  ergodic  Markov  chain;  thus,  {zi}  is  nonde¬ 
creasing  and  approaches  1/p  as  a  limit.  Thus,  we  conclude  that  ipi(z)  >  0  for  all 
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z  G  [1/p,  1)  and  for  each  i  >  2.  Hence,  Kaplan’s  condition  is  satisfied.  To  prove 
the  existence  of  a  positive  integer  N  such  that  d(i)  >  0  for  every  i  >  N,  we  observe 
that,  for  each  e  >  0,  we  have  A p  =  |pW  —  p|  <  £  whenever  i  >  Ne  for  some  Ne  >  0. 
By  our  hypothesis,  p  >  1,  so  we  set  e  =  p  —  1.  Removing  the  absolute  value  in  A p, 
we  obtain 

1-p  <  PW  -  P  <  P-1,  i>Np_  i,  (4.16) 

which  shows  that  pW  >  1  for  i  >  jVp_ i.  By  equation  (4.8),  we  now  see  that  d(i)  >  0 
for  i  >  Np_i,  and  thus,  by  Theorem  1  of  [95],  we  conclude  that  the  QBD  is  not 
positive  recurrent,  and  hence  not  ergodic.  Therefore,  it  is  necessary  that  p  <  1  for 
ergodicity.  □ 

Although  Theorem  4.2  applies  to  discrete-time  systems,  it  may  be  easily  ex¬ 
tended  to  the  continuous-time  version  using  an  embedded  discrete-time  Markov 
chain.  By  considering  the  Markov  chain  embedded  at  level  jump  epochs  in  our  unre¬ 
liable  retrial  queueing  model,  we  directly  apply  Theorem  4.2  to  obtain  the  condition 
required  to  ensure  stability  of  the  queueing  system. 

Theorem  4.3.  An  irreducible,  aperiodic,  continuous-time,  level- dependent  QBD  is 
ergodic  if  and  only  if 

p  =  lim  pW  <  1,  (4.17) 

i—>  oo 

with  ck*)  defined  as  the  unique  positive  solution  to  the  system  of  equations  given  by 
ot{l)Ail)  =  0,  awe  =  1,  i  =  0, 1, 2, ... , 

where  A^  =  +  A^  +  A^  and  e  is  a  column  vector  containing  ones. 

Proof.  This  assertion  is  a  direct  consequence  of  Theorem  4.2  and  Theorem  7.2.4  of 
[66]  (page  158).  □ 

Theorem  4.3  gives  us  the  means  by  which  we  may  characterize  the  stability  for 
the  queueing  process  {(R(t),  Z(t),X(t)  :  t  >  0)}. 
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Theorem  4.4.  The  QBD  with  infinitesimal  generator  matrix  Q*  is  positive  recurrent 
if  and  only  if 


_  -e'[(Q  -  A(ct)  )A(|)  +  (Q^A(a+|))A(A)]g;-e 
p-  e'(Q^A(a))(Q-A(u  +  $))D-‘e  ’  '  ; 

where  Da  =  Q  —  A  (a:  +  £)  and  e  is  a  column  vector  containing  ones  as  entries. 

Proof.  We  begin  with  the  derivation  of  the  sequence  (pb)  :  i  =  1,  2, . . .}  for  the  QBD 
with  infinitesimal  generator  Q*.  In  order  to  facilitate  the  computation  of  the  limit 
over  i,  we  purposely  exclude  the  degenerate  situation  in  which  a^A^e  =  0,  thus 
making  the  sequence  (pb)}  well-defined  for  each  i.  It  is  now  possible  to  apply  (4.7) 
to  the  elements  of  the  QBD  defined  in  (4.2),  upon  which  we  obtain 

(l)  -e'A{\)A(Z)  +  (Q-A(a))(iA{Q)+A(\))A(\  +  Z)D£)Dz1e 

/  e'iA(e)(Q~A(a))(Q-A(pc  +  ^))D^D^e  J 

where  =  (Q  —  iA(0)  —  A(A  +  pi  +  £) ).  Using  standard  techniques  for  the 
determination  of  limits  of  rational  functions,  the  limit  of  (4.19)  as  i  — >  oo  gives  us 
(4.18),  as  desired.  Note  that  this  result  implies  the  existence  of  the  limit  for  all 
choice  of  parameters,  excluding  those  that  result  in  the  degenerate  case  discussed  at 
the  beginning  of  this  proof.  □ 

When  there  is  only  a  single  environment  state,  our  queueing  model  corresponds 
to  the  model  of  Kulkarni  and  Choi  [62]  with  exponentially  distributed  service,  in¬ 
terfailure,  and  repair  times.  In  such  case,  the  expression  for  p  in  (4.18)  reduces  to 
Equation  (2.4)  of  [62], 

4-3  Performance  Measures 

Approximate  performance  measures  for  the  queueing  system  Z(t),X(t))  : 

t  >  0}  in  steady-state  may  be  obtained  with  numerical  results  for  the  steady-state 
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distribution,  which  is  given  by  the  vector  7r  =  [n (i,j,  k)},  where 

7T (i,j,  k)  =  lim  P{R(t)  =  i,  Z(t)  =  j,X(t )  =  k}  ,  (i,j,  k)  G  5. 

t — >oo 

We  begin  with  a  discussion  of  the  expected  size  of  the  retrial  orbit  R  in  steady-state, 
which  is  given  by 

OO 

E[R]  =  ^i(7Tie).  (4.20) 

1=1 

Since  the  steady-state  probabilities  correspond  to  each  state  of  the  trivariate  repre¬ 
sentation  of  the  queueing  process,  we  must  aggregate  these  probabilities  in  such  a 
way  as  to  obtain  the  probability  of  being  in  orbit;  in  other  words,  for  the  sequence 
{pi  :  i  >  0}  of  steady-state  probabilities  of  the  orbit  size  i,  we  obtain  pt  =  7vte. 

Many  other  performance  measures  may  be  obtained  in  much  the  same  way  as 
was  done  for  the  expected  size  of  the  orbit.  For  instance,  let  us  consider  the  limiting 
proportion  of  time  that  the  system  is  under  repair.  For  each  orbit  size  i,  we  must 
find  all  possible  phases  for  which  X  =  0;  in  this  case,  the  long-run  proportion  of 
time  that  the  system  is  down  is  given  by 

oo  m 

P0  =  P (System  is  Down)  =  EE  <i,3, 0)-  (4.21) 

i= 0  j= 1 

We  may  therefore  conclude  that  the  proportion  of  time  the  server  spends  in  state 
k,  k  =  0, 1,  2,  is  given  by 

oo  m 

P«  =  EE7^,-^),  k  =  0,1,2.  (4.22) 

i=0  j= 1 

The  expected  number  in  system  E[L\  and  system  sojourn  time  i£[W]  follow 
directly  from  the  orbit-size  probabilities  and  according  to  the  following  logic.  We 
base  our  approach  to  the  computation  of  E[L]  upon  the  observation  that  the  system 
size  always  equals  the  orbit  size  whenever  the  system  is  down  or  the  server  is  idle. 
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When  the  server  is  busy,  one  is  added  to  the  expected  number  E[R]  in  orbit.  Thus, 
we  have 


E[L\  =  E[R](P0  +  P1)  +  (E[R\  +  1)P2 
=  E[R]  +  P2. 

We  next  turn  our  attention  to  the  computation  of  A  [IF] .  For  work-conserving  sys¬ 
tems,  the  typical  approach  is  to  employ  Little’s  Law,  which  relates  E[L\  to  E [IF] . 
It  is  given  by  the  well-known  formula 

E[L\  =  A  E[W],  E[R]  =  XE[Wr], 

where  A  is  the  average  rate  of  arrivals  to  the  system  in  steady  state.  Given  the 
equilibrium  probability  vector  p  =  \pj  :  j  =  1, . . . ,  m]  of  the  random  environment, 
we  obtain  the  average  arrival  rate  A  =  pAe,  and  then  solve  for  E [IF] : 

E[W]  =  E[L\/ A,  E[Wr]  =  E[R]/ A.  (4.23) 

4-4  Useful  Algorithms 

As  a  result  of  the  recursive  elements  of  many  of  its  defining  analytical  results, 
QBD  methods  do  not  readily  admit  closed-form  solutions  for  the  steady-state  distri¬ 
bution.  There  do  exist  certain  classes  of  level-independent  QBD  processes  for  which 
an  explicit  rate  matrix  R  may  be  derived;  for  this,  we  refer  the  reader  to  [108].  Nev¬ 
ertheless,  the  model  that  we  study  in  this  chapter  is  not  amenable  to  the  methods 
described  in  [108],  and  so  we  must  resort  to  algorithmic  techniques  for  the  compu¬ 
tation  of  an  approximate  stationary  distribution.  In  this  section,  we  shall  review 
the  algorithms  of  Bright  and  Taylor  [25]  and  discuss  their  implementation.  In  the 
section  that  follows,  we  shall  compute  some  important  measures  of  interest  using  the 
approximated  steady-state  distribution. 
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In  discussing  the  algorithms  of  this  section,  we  shall  refer  to  the  continuous¬ 
time  level-dependent  QBD  (LDQBD)  as  the  bivariate  process  {(A"(f),  Y(t))  :  t  >  0}. 
Nevertheless,  it  should  be  clear  to  the  reader  that  we  may  substitute  for  this  a  multi¬ 
layered  (i.e.  n- variate  for  n  >  3)  LDQBD  process  if  so  desired.  It  should  also  be 
mentioned  that  the  more  general  case  of  the  LDQBD,  in  which  the  number  of  phases 
Mi  corresponding  to  level  i  for  %  —  0, 1,  2, . . .  may  vary,  is  assumed  in  [25].  We  will 
state  Theorem  4.2  with  this  assumption  intact.  However,  we  will  henceforth  assume 
that  M  =  Mi  =  Mj  <  oo  for  every  i,j  G  Z+;  in  other  words,  the  QBD  will  be 
homogeneous  in  the  number  of  phases  at  each  level. 

It  has  already  been  mentioned  that  the  existence  of  a  limiting  distribution 
hinges  upon  the  behavior  of  the  QBD  at  level  0.  It  is  therefore  fortunate  that 
the  process  restricted  to  level  0  of  a  continuous-time  QBD  is  itself  a  CTMC  that 
possesses  a  time-to-absorbtion  (transition  to  the  next  level)  that  is  PH-distributed 
with  representation  (ab0\Q(°)),  where  we  define  =  A®  +  Aq^G^,  and  a®  is 
the  positive  vector  solution  to  the  system 

M 

a(0)Q(0)  =  0,  =  I- 

3=0 

As  always,  we  assume  that  there  are  M  <  oo  states  (phases)  in  level  0  and,  further¬ 
more,  it  is  clear  that  Q is  the  infinitesimal  generator  of  the  restricted  process. 

The  statement  of  Theorem  3.4  asserts  the  necessary  and  sufficient  conditions 
for  the  ergodicity  of  a  level-dependent  QBD  process.  However,  setting  these  criteria 
to  the  task  of  obtaining  a  numerical  steady-state  distribution  entails  the  solution  of 
a  system  of  linear  equations  for  each  level  i,  which  in  turn  would  require  explicitly- 
defined  matrices.  Since  QBDs  that  possess  such  explicitly-defined  G  matrices 
are  few  and  far  between,  this  would,  at  best,  be  deemed  an  untenable  approach. 
Nevertheless,  as  in  [25],  Theorem  3.4  may  serve  as  the  basis  for  efficient  algorithms 
dedicated  to  obtaining  the  steady-state  distributions  of  level-dependent  QBDs. 
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The  conditions  for  the  existence  of  a  stationary  distribution  of  a  continuous¬ 
time  level-dependent  QBD,  which  is  that  the  vector  of  stationary  probabilities  7r  be 
the  unique  solution  to  the  system  of  equations  given  by  (3.17)  and  (3.18),  clearly 
shows  that  the  rate  matrix  R^\  i  —  0, 1,  2, . . is  key  to  its  determination.  The  algo¬ 
rithms  developed  by  Bright  and  Taylor  [25]  are  well-suited  to  the  purpose  of  comput¬ 
ing  7r  using  standard  mathematical  computing  environments  such  as  MATLAB®,  as 
we  have  done  for  the  unreliable  retrial  queue  in  a  random  environment.  The  funda¬ 
mental  idea  behind  this  computational  procedure  is  based  upon  the  following  result, 
which  is  given  as  Lemma  1  in  [25]: 

Lemma  4.2.  (Bright  and  Taylor)  If  the  level- dependent  QBD  given  by  {(X(t),  Y(t))  : 
t  >  0}  with  state  space  {  (i,j)  :  i  >  0,  1  <  j  <  Mj }  is  positive  recurrent,  then  the 
sequence  {Ri-i  =  0,1,2,...}  is  given  by 

OO  k—  1 

Ri  Y  1  '  n  Di+r->  >  *  >  (4-24) 

k= 0  r=0 

where  U\  and  D\  are  Mj_i  x  M*_ i+2r  and  M*_i  x  M,;_i_2r  matrices  respectively  and 
are  given  by  the  recursive  expressions 


U?  =  Aj^-A^)-1  for  k  >  1,  (4.25) 

A9  =  4°(-4i“1))“1  fork>  1,  (4.26) 

Ui+  —  U\  Ui+2r[l  ~  U(+2r+lDi+3.2r  ~  -D[+2r+1  Uf+2r  ]"\  (4.27) 

Dr+1  =  DfDT  [/  _  (4-28) 


Lemma  4.2  is  the  focal  point  of  a  series  of  four  algorithms  whose  overall  ob¬ 
jective  is  to  compute  an  approximate  steady-state  distribution.  One  may,  of  course, 
repeatedly  apply  Lemma  4.2  to  find  each  rate  matrix,  but  this  is  not  necessary  due 
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to  the  following  relationship: 


4°  +  RtA{;+1)  +  RilR^A^]  =0,  i  >  0,  (4.29) 

which  allows  Ri  to  be  dehned  recursively  as 

Ri  =  4°[-4<+1)  -  i2i+i4<+2)]-1-  (4-30) 

Thus,  it  is  more  efficient  to  obtain  an  approximation  for  RK_ 2  for  some  threshold 
level  K  and  then  work  backwards  using  equation  (4.30)  to  obtain  all  of  the  remaining 
rate  matrices.  Then,  after  solving  the  system  in  Theorem  3.1  to  determine  7 r0(/l), 
the  remaining  vectors  7 Tq(iF),  i  >  0  may  be  obtained  through  successive  applications 
of  equation  (3.5)  while  normalizing  the  set  of  probabilities  with  each  iteration  i.  This 
notion  provides  the  basis  for  Algorithm  1  in  Figure  4.3,  which  is  due  to  Bright  and 
Taylor  [25] .  This  defines  the  overall  algorithm  that  defines  our  method  of  approaching 
the  computation  of  the  approximate  steady-state  probabilities.  Several  of  the  key 
steps  of  this  algorithm  are  addressed  in  further  detail  as  sub-algorithms,  each  of 
which  likewise  appeared  in  the  article  by  Bright  and  Taylor  [25]. 

Step  (2)  of  Algorithm  1  is  rather  involved,  as  it  requires  an  iterative  application 
of  Lemma  4.2.  The  purpose  of  the  algorithm  is  to  obtain  an  approximation  for  RK_r 
based  upon  some  predefined  tolerance  level  e  >  0  that  dictates  the  number  of  terms 
of  the  sequence  {RK-i(N)  :  IV  >  1}  dehned  by 

N  k—  1 

Ri{N)  =  J2Ui  II Dri+2r~-1,  h  N  >  0  (4.31) 

k= 0  r= 0 

that  we  must  obtain  for  a  given  threshold  K  >  0.  In  other  words,  Ri(N)  is  the  IV  + 1- 
th  partial  sum  of  (4.24).  We  thus  iterate  through  successive  values  of  Rk^^N),  N  > 
1  until 

[  Rk_1(N)  -  R^N  -  1)  |oo  <  e,  (4.32) 
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Algorithm  1  (Bright  and  Taylor  [25]): 


1.  Determine  threshold  level  K. 

2.  Obtain  RK-i  using  Algorithm  2. 

3.  Obtain  the  remaining  Rj,  0  <  i  <  K  —  2  using  (4.29). 

4.  Solve  the  following  for  -kq (K): 

vr  o(K)(A^  +  R0A vr  0(K)e  =  1. 

5.  for  i  =  1  :  K 

Compute  7 Ti(K)  =  7Tj_i(A)i?j_i. 

Normalize  {-n^K)  :  k  =  0,1,...,  A"}  so  that  Ylk=o  7r^  =  1- 

Figure  4.3  Top-level  algorithm  for  computing  the  steady-state  distri¬ 
bution  {7t,:(A')  :  0  <  i  <  K  —  1}. 


where  (AIoq  =  maxjj  A  e  ]^mxn  js  the  L°°  norm.  This  results  in  Algorithm 

2,  which  is  depicted  in  Figure  4.4.  Since  Algorithm  2  involves  the  computation  of 
the  C/f  and  fc+1  matrices,  which  are  requisite  for  obtaining  the  N-th  partial  sum 
Rk- i(iV),  we  must  define  yet  another  subprocedure,  namely  Algorithm  3,  in  order  to 
wholly  specify  the  procedure  given  in  Algorithm  2.  We  shall  discuss  the  elaboration 
of  this  next  task  in  the  paragraph  that  follows. 

For  convenience,  we  now  define  the  UD-pair  UD(N,i )  as  the  ordered  pair 
(A.A+  2.«)  whose  constituent  matrices  appear  in  Step  (2)  of  Algorithm  2.  Based 
on  the  recursive  definitions  of  these  matrices  in  (4.25)-(4.28),  it  might  seem  that  an 
algorithm  to  compute  the  UD-pair  need  only  consist  of  their  repeated  evaluation, 
and,  as  such,  is  straightforward  in  execution.  However,  implementing  these  recursive 
relations  directly  will  most  likely  result  in  the  repeated  computation  of  the  same 
U D-pairs,  which  is  clearly  undesirable.  The  algorithm  obtains  and  stores  only  those 
pairs  UD(rj,i )  that  are  necessary  to  complete  the  recursion  up  to  UD(N,i),  N  > 
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Algorithm  2  (Bright  and  Taylor  [25]): 


1.  Initialize  the  following  variables: 

Set  N  =  0.  \ 

n  n  i  Steps  1  and  2  of  Algorithm  3. 

Set  U  =  U?,D  =  D°+  2.  J 

Set  II  =  I  (Identity  matrix). 

Set  Ri( 0)  =  U. 

2.  Increment  N:  N  =  N  +  1. 

3.  Make  the  following  assignments: 

n  =  d  •  n. 

1/  =  £/A,  D  =  D”2„+1.  }  Obtained  from  Algorithm  3 
Ri(N)  =  Ri(N  -1)  +  U-U. 

4.  If  | Ri(N)  —  Ri(N  —  1)|  >  e,  then  go  back  to  step  (2). 

5.  Set  R  =  Ri(N). 


Figure  4.4  Procedure  to  compute  the  rate  matrix  RK_ i  in  step  (2)  of 
Algorithm  1. 


0.  Fortunately,  the  relationships  among  UD- pairs  are  regular,  and  thus  may  be 
represented  pictorially  as  in  [25:  Fig  2].  The  algorithm  itself  appears  as  Figure  4.5. 

Finally,  and  perhaps  most  critically,  is  the  question  of  how  to  choose  the  maxi¬ 
mum  level  K  so  as  to  ensure  the  validity  of  the  approximate  steady-state  probability 
vector  7r  (K).  The  obvious  criterion  would  be  to  set  K  large  enough  so  that  the  sum 
of  all  steady-state  probabilities  is  close  to  1.  Putting  this  idea  into  practice,  how¬ 
ever,  is  difficult  if  one  desires  an  a  priori  answer  -  which  is  essential  if  one  wishes  to 
avoid  the  brute-force  alternative  of  running  the  algorithm  until  the  desired  accuracy 
is  achieved.  What  further  makes  this  idea  so  unattractive  is  the  explosion  of  the 
number  of  f/D-pairs  -  and  hence  matrix  inverses  -  that  one  needs  to  compute  as 
the  values  of  K  increase.  This  situation  is  readily  apparent  if  one  observes  the  rapid 
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Algorithm  3  (Bright  and  Taylor  [25]): 


1.  if  N  =  0  then  compute  UD(0,i )  and  store. 

2.  for  m  =  0  to  N 

for  n  =  K  +  (2N~n+1  -  1)2* 

to  K  +  (2N~n+1  -  l)2i+1  step  2n 
Evaluate  UD(m,n)  and  store. 

3.  Increment:  N  =  N  +  1. 

4.  Using  all  UD- pairs  stored  thus  far,  compute  UD(N,  i )  and  store  it  as 
well. 

5.  Remove  all  stored  C/D-pairs  except  UD(n,i  +  (2N~m  —  l)2-7+1),  j  = 

0,1,..., A. 

6.  Return  to  Step  2  if  Algorithm  2  has  not  yet  terminated. 

Figure  4.5  Procedure  to  compute  UD(N,i)  for  the  Algorithm  2  (c.f. 
Fig.  4.4). 


growth  of  the  number  of  UD-pairs  contained  within  the  outer  trapezoidal  region  of 
the  graph  in  [25:  Fig  2],  In  order  to  counter  this  difficulty,  a  heuristic  of  sorts  is 
presented  in  the  form  of  Algorithm  4  in  [25].  Simply  put,  it  is  a  reformulation  of 
Algorithm  1  with  the  option  to  increase  the  value  of  K  until  nKe  <  e,  for  some 
arbitrarily  chosen  e  >  0.  The  algorithm  is  given  in  Figure  4.6. 

As  noted  in  [25],  it  is  theoretically  possible  to  have  a  level-dependent  QBD 
whose  matrices  are  not  parametrically  defined  (that  is,  explicitly  dependent 
upon  level  i  >  0  and  a  finite  number  of  parameters).  In  this  case,  the  application 
of  the  algorithms  discussed  in  this  section  requires  the  input  of  a  large  number  of 
unrelated  Ajy  matrices  into  the  computer  in  order  to  reach  the  required  tolerance 
for  the  smallest  steady-state  probability.  Even  if  this  were  done,  however,  the  unpre¬ 
dictability  of  the  QBD  generator  blocks  for  a  nonparametric  QBD  would  make  this 
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Algorithm  4  (Bright  and  Taylor  [25]): 


1.  Set  I\prev  =  0  and  choose  some  tolerance  e  >  0. 

9  'sot  K  '>  T\ 

±*-new  v  prev  • 

3.  Compute  the  partial  sum  -Rjr„ei„-i(4)  of  expression  (4.24). 

4.  if  |f?j(4)  —  f?j(3)|  >  e  then  go  back  to  Step  (2). 

5.  Compute  RKnew-2,  Ri<new-3,  ■  ■  ■ ,  RKprev  using  the  recursive  definition 
(4.30)  of  the  rate  matrix  R{. 

6.  Solve  the  system 

7To(A',ieu,)(T^0)  +  RqA^)  =  0,  7T  0(Knew)e  =  1. 

7.  for  i  =  Kprev  +  1  to  Knew 

( R-  new )  — 1  (■Rnew')Ri — 1* 

Normalize  n0(Knew),  7ri(Aneu;), . . . ,  7Vi(Knew) 
so  that  J2k=o  ^k(Knf,w)e  =  1. 

8  •  Set  I£prev  -Rnew  * 

9.  if  TT Knew{Knew)e  >  e  then  go  back  to  Step  (2). 

10.  Set  K  =  Krrrf„„ . 


Figure  4.6  Procedure  to  compute  ir(K)  for  some  suitably-chosen 
K  >  0. 


determination  tricky  at  best.  It  is  therefore  recommended  in  [25]  that,  even  in  the 
parametric  case,  one  should  take  great  care  in  determining  stopping  criteria  when 
applying  the  algorithms. 

4-5  Busy  Period  Analysis 

We  approach  the  study  of  the  busy  period  of  the  system,  as  in  Section  3.1.3,  in 
terms  of  the  fundamental  period  of  the  system.  For  the  QBD  process  of  this  chapter, 
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we  may  formally  express  the  distribution  function  G°(t)  of  the  busy  period  as  the 
m  x  m  matrix 


[G°{T)\kk,  =  P{r  <  oo,  (Y(r),  Z{t),X{t))  =  (0, 1,  k') 

■  ■  ■  I  (Y(0),  Z(0),X(0))  =  (1,  2,  k)}, 


where  k,  k!  G  {1, ... ,  m}, 


\R(t)  +  l  if  X (t)  =  2 

Y(t)  =  <  and  (R(t),Z(t),X(t))  G  5. 

I  R(t)  otherwise. 

A  complicating  issue  is  that  failures  of  the  server  may  preclude  the  servicing  of 
customers  in  a  busy  period;  in  addition,  the  server  of  a  retrial  queue  may  be  up  and 
idle  with  customers  still  in  orbit,  which  is  likewise  before  the  termination  of  the  busy 
period,  ffence,  it  is  more  instructive  to  study  the  fundamental  period  corresponding 
to  level  i,  whose  distribution  (matrix)  function  is  denoted  by  [G^(x)]jjies,  i  £  2A- 

Application  of  the  block  matrix  components  (4.3)  through  (4.5)  of  the  QBD 
representation  of  the  queue  {(R(t),  Z(t),X(t))  :  t  >  0}  to  Theorem  3.10  results  in  a 
matrix  quadratic  equation  which,  when  solved,  gives  the  transform  G^(z,s)  of  the 
matrix  distribution  of  the  fundamental  period.  For  the  purpose  of  simplification,  let 


Xi{z,s)  =  G(l)(z,s). 


We  thus  obtain  the  equation 

X«(*.  *)  =  (*A(0)  C,  +  x?(*. »)<??>)  V.  (4.33) 
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where,  if  we  define  T\  =  Q  —  A(A), 


C,= 


0 

0 

0 


0  (A (a)  -  Tx)  (A(£  +  /x)  -  Tx) 
0  A(a)(A  (|  +  M)-7a) 

0  A(/i)  (A (a)  —  T\) 


0  A(0A(A)(A(^  +  M  +  a)-2TA) 

0  A(A)  {T2  —  [A(id)  +  A(£)  A(/j,  +  e)]  Tx 

+  [A(£  +  iO)  +  A(0  A(a)  -  A(A)  A(/x)]} 
0  A(O{T2-A(a  +  |  +  z0)  +  A(A)AQu)} 


A(A)2  (A(a)  —  TA) 
A(A)2A(«) 

A(A)  [T2  -  A(a  +  I  +  id)  Tx  +  A(i0)  A(a)] 


for  i  >  0,  and 


Tj  —  —  +  A(«+2£+/i+i0)  T2 

—  [A(«+£+*0)  A(£+/i)  —  A(A)  A(/i)]  TA 

+  A  (a)  [*A(0)  A(£+fi)  -  A(A)  A(/x)] . 


The  matrix  G  of  the  fundamental  period  is  thus  the  smallest  positive  solution  to 
(4.33).  The  necessary  and  sufficient  conditions  for  the  (exact)  solvability  of  matrix- 
quadratic  equations  may  be  found  in  [98].  However,  due  to  the  complexity  of  the 
foregoing  expression,  a  closed-form  expression  for  the  transform  may  not  be  derived, 
and  so  we  must  resort  to  an  approximation  using  either  the  simple  algorithm  (3.36) 
or  by  obtaining  the  matrices  through  computation  of  the  steady-state  distribution 
via  the  algorithms  that  were  presented  in  Section  4.4. 
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4-6  Numerical  Illustrations 


We  conclude  this  chapter  with  numerical  illustrations  of  the  results  we  have 
obtained  for  the  queueing  system  {(R(t),  Z(t),X(t))  :  t  >  0}.  Note  that,  in  addition 
to  the  overall  traffic  intensity  p  =  lim^^  pb)  which  is  derived  from  Theorem  4.2,  we 
may  compute  the  traffic  intensity  pj  for  the  system  when  the  environment  is  in  state 
j  as  follows: 


Pj 


+  ajiXj  +  £j) 

c^jiP'j  T  £ j ) 


jeS. 


(4.34) 


The  traffic  intensity  (4.34)  may  be  obtained  through  simplification  of  the  traffic 
intensity  model  of  [62]  or  by  fixing  the  parameters  of  the  model  for  m  >  2  and  then 
applying  (4.7)  and  (4.10). 


4-6.1  Example  1:  Three- State  Environment 

We  first  consider  a  system  operating  in  a  three-state  random  environment 
whose  infinitesimal  generator  is  given  by 


Q  = 


-1.0 

1.6 

0.0 


1.0 

-2.6 

3.2 


0.0 

1.0 

-3.2 


(4.35) 


Table  4.1  summarizes  the  system  parameter  values.  Notice  that  the  traffic  intensity 
for  state  1  of  the  environment  exceeds  unity,  and,  thus,  the  system  may  experience 
periods  of  instability  whenever  the  environment  is  in  state  1.  Nevertheless,  the 
overall  traffic  intensity  obtained  by  (4.7)  and  (4.10),  is  p  =  0.4730  <  1,  so  it  is 
(overall)  a  stable  system. 


Table  4.2  lists  the  first  20  equilibrium  probabilities  of  orbit  size.  A  graphical 
depiction  of  these  probabilities  -  together  with  an  additional  56  probabilities  -  is 
shown  in  Figure  4.7.  The  probabilities  represented  in  this  figure  were  computed 
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Tabic  4.1  Summary  of  parameters  for  3-state  example. 


Environment  State  (j)  A j 

N 

0  aj  Qj 

Pj 

1 

1.0 

0.5 

9.0  4.0  1.5 

1.2895 

2 

0.1 

0.8 

2.0  7.0  3.0 

0.7602 

3 

3.0 

10.0 

0.1  2.0  5.0 

0.3218 

Table  4.2 

Steady-state  probabilities  of  orbit  size 

for 

Example  1. 

i 

Pi 

i 

Pi 

0 

0.02798110 

11 

0.03540854 

1 

0.03306333 

12 

0.03376359 

2 

0.03706997 

13 

0.03208472 

3 

0.03973863 

14 

0.03039734 

4 

0.04131836 

15 

0.02872157 

5 

0.04205032 

16 

0.02707325 

6 

0.04212703 

17 

0.02546462 

7 

0.04169928 

18 

0.02390498 

8 

0.04088593 

19 

0.02240120 

9 

0.03846185 

20 

0.02095816 

10 

0.03698781 

using  Sfc=o7r(b-b  f°r  eac^  *  e  {0,1,2,...}.  Note  the  geometric  rate  of 
decay  exhibited  by  the  steady-state  distribution,  which  is  a  defining  characteristic 
of  an  ergodic  system. 

The  efficient  recursive  algorithms  of  Bright  and  Taylor  [25],  are  useful  for  com¬ 
puting  the  steady  state  probabilities  of  level-independent  QBDs  and  may  be  adapted 
to  our  model.  Here  we  make  use  of  Algorithms  1  through  4  of  [25]  to  obtain  (approx¬ 
imate)  steady-state  probabilities  from  which  we  may  compute  queueing  performance 
measures.  Let  TTiJ  k(t)  =  P(R(t )  =  i,  Z(t)  =  j,X(t )  =  k )  and  define  as  before 

7T =  lim  7riJifc(t),  (■ i,j,k)eS . 

r— xx) 
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Figure  4.7  Pictorial  representation  of  steady-state  orbit-size 
probabilities  for  Example  1. 

The  limiting  probability  that  %  customers  are  in  the  orbit  is  given  by 

3  2 

7Ti  =  J]^7r(i,j,/c),  i  >  0. 
j= 1  k= 0 

The  steady-state  expected  number  of  customers  in  the  orbit  (E[R])  is  approximated 
(using  the  first  76  probabilities)  by  E[R]  =  ^*=o  1 T  ~  15.6684.  We  obtain  the 
overall  system  size  (E[L\)  by  summing  the  expected  number  in  orbit  and  the  ex¬ 
pected  number  in  service,  namely,  E[L\  =  E[R\  +  P2  ~  16.0921,  where  P2  = 
Si=o  X/j=i  7T (i,j,  2)  cs  0.4237  is  the  long-run  probability  that  the  server  is  not  failed 
and  busy.  Applying  Little’s  law  and  the  average  arrival  rate  (A  =  A p  «  0.90558), 
we  respectively  obtain  the  expected  time  spent  in  the  retrial  orbit  and  in  the  system 
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by 


E[Wr]  =  E[R\/X  «  17.302,  E[W]  =  E[L]/X  «  17.77. 

^ .  6'.  I?  Example  2:  Seven-State  Environment 

We  shall  next  consider  a  7-state  example  that  exhibits  interesting  non-intuitive 
behavior  as  compared  to  the  previous  3-state  example.  As  before,  we  define  the  envi¬ 
ronment  by  its  infinitesimal  generator  One  might  observe  that  the  rates  of  transition 
to  states  2  and  3  are  larger  (in  general)  than  that  of  any  of  the  other  transitions. 
From  the  table  of  system  parameter  values  shown  in  Table  4.3,  we  may  further  ob¬ 
serve  that  the  queue  is  subject  to  heavy  traffic  when  the  environment  is  in  states 
2  and  3.  The  values  of  these  parameters  were  set  in  order  to  observe  the  effects  of 
heavy  traffic  upon  the  stability  of  the  system. 

-7.6  2.0  3.0  1.0  1.0  0.1  0.5 

0.5  -8.0  2.5  1.0  2.0  0.8  1.2 

0.3  1.5  -0.8  1.0  1.0  1.0  1.0 

Q  =  2.0  3.0  5.0  -1.2  0.1  1.0  0.1  •  (4-36) 

0.8  2.5  2.0  1.1  -7.9  0.7  0.8 

1.5  1.0  1.6  1.2  0.5  -6.3  0.5 

2.0  2.5  2.0  1.0  1.8  0.9  -10.2 

Next,  through  the  employment  of  the  algorithms  of  Bright  and  Taylor,  we 
computed  the  steady-state  probabilities  corresponding  to  the  retrial  queueing  system 
thus  defined,  from  which  we  subsequently  obtain  the  probabilities  of  orbit  size  (c.f. 
Table  4.4). 

The  values  of  the  various  system  performance  measures  is  shown  in  Table  4.5. 
From  this  we  can  see  that,  although  the  greater  stability  of  the  system  with  a  7-state 
environment  is  not  reflected  in  orbit  or  system  sizes,  it  is  definitely  present  in  the 
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Table  4.3  Parameters  for  the  system  of  Example  2. 


Environment 
State  j 

Parameter 

Pi 

A, 

llj 

0 

Olj 

1 

3.0 

7.0 

0.5 

2.0 

1.0 

0.5667 

2 

1.0 

3.0 

1.1 

0.5 

11.0 

1.0488 

3 

3.0 

12.5 

1.5 

0.5 

2.0 

0.9643 

4 

2.0 

12.5 

4.0 

2.0 

2.0 

0.6061 

5 

2.0 

4.5 

1.0 

6.0 

5.0 

0.6061 

6 

0.5 

2.0 

0.7 

3.0 

1.0 

0.4877 

7 

0.5 

4.0 

1.5 

0.5 

0.5 

0.6364 

P 

0.1915 

Table  4.4  Steady-state  probabilities  for  orbit  sizes  up  to 
and  including  i  =  20  for  the  model  presented 
in  Example  2. 


i 

Pi 

i 

Pi 

0 

0.01470817 

11 

0.03366217 

1 

0.01887110 

12 

0.03322658 

2 

0.02262905 

13 

0.03261428 

3 

0.02577557 

14 

0.03185676 

4 

0.02832568 

15 

0.03098201 

5 

0.03032339 

16 

0.03001475 

6 

0.03182163 

17 

0.02897667 

7 

0.03287514 

18 

0.02788673 

8 

0.03353730 

19 

0.02676134 

9 

10 

0.03385857 

0.03388586 

20 

0.02561467 
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Tabic  4.5  Performance  measures  for  Examples  1  and  2. 


Model 

Performance  Measure 

Rmax 

P 

E[Lr] 

E[L) 

E[Wr] 

E[W] 

3-State 

15.668 

16.092 

17.302 

17.770 

7 

0.4730 

7-State 

19.550 

19.825 

7.680 

7.788 

10 

0.1915 

comparison  between  sojourn  times.  The  values  for  times  in  orbit  and  in  system  are 
less  than  half  of  what  they  are  for  the  system  with  a  three-state  environment.  The 
interpretation  that  one  may  ascribe  to  this  behavior  is  that  system  size  is  kept  in 
check  -  despite  heavy  traffic  -  by  the  relatively  short  times  that  the  average  customer 
spends  in  orbit  and  in  service. 

4-6.3  Comparison  to  Simulated  Data 

It  was  mentioned  at  the  beginning  of  this  chapter  that  the  effective  times  of  ser¬ 
vice,  arrivals,  breakdowns,  repairs,  and  retrials  will  significantly  affect  how  one  mod¬ 
els  the  retrial  queueing  system.  Indeed,  one  must  effectively  assign  PH- distributed 
durations  to  these  activities,  the  capability  of  which  is  not  inherently  present  in  the 
graphical  process  simulator  Arena®.  As  a  consequence,  two  basic  work-arounds  need 
to  be  employed  depending  upon  whether  the  activity  is  a  “service1' -type  activity 
or  an  “arrival” -type  activity.  Let  us  first  discuss  the  procedure  for  simulating  the 
modulation  of  arrivals  to  the  system.  Suppose  that  the  environment  has  undergone 
a  transition  from  state  i  to  state  j.  Simply  changing  the  arrival  rate  will  not  work 
since  all  scheduled  arrivals  at  the  previous  rate  need  to  be  instantiated  before  the  new 
rate  takes  effect.  This  results  in  a  backlog  of  customers  from  the  arrival  distribution 
of  the  previous  environmental  state,  thus  invalidating  the  statistics  for  the  process 
under  the  new  environmental  state.  It  turns  out  that,  in  order  to  deal  with  instan¬ 
taneous  rate  changes  for  arrivals,  it  is  necessary  to  set  the  exponential  arrival  rate 
to  a  constant  \max  =  max*^ t  my  (A j),  and  then  allow  only  the  proportion  Aj/Amaa. 
of  customers  to  enter  the  system. 


Services,  on  the  other  hand,  must  be  dealt  with  in  a  different  manner  since  a 
customer  in  service  has  already  been  instantiated,  as  opposed  to  arriving  customers 
whose  future  instantiation  is  affected  by  the  exponential  rate  of  arrival.  These  cus¬ 
tomers  must  have  their  current  service  preempted  and  they  must  then  be  reassigned 
a  full  exponential  service  time  of  rate  Hj  (due  to  the  memorylcssness  of  the  expo¬ 
nential  distribution).  Breakdowns  are  treated  as  arrivals  to  the  system  while  repairs 
are  dealt  with  in  exactly  the  same  way  as  services.  The  distribution  of  retrials  is 
perhaps  the  most  difficult  to  simulate.  If  there  are  k  customers  in  orbit  during 
the  environmental  transition,  then  there  are  two  ways  to  proceed.  One  may  either 
(1)  pick  an  estimated  maximum  orbit  size  kmax  and  allow  the  ratio  9j/9max  of  cus¬ 
tomers  from  an  exponential  retrial  rate  of  kmax9max  to  proceed,  or  (2)  preempt  every 
customer  in  the  retrial  orbit  and  reassign  the  appropriate  exponential  inter-retrial 
duration.  Each  method  has  its  difficulties,  but  (2)  may  be  the  preferred  option  for 
moderate-size  environments  due  to  the  lack  of  the  need  to  estimate  a  maximum  orbit 
size,  thereby  saving  memory  and  eliminating  the  need  to  produce  large  numbers  of 
unneeded  entities. 


We  will  compare  the  average  performance  measures  of  250  replications  of  Arena 
simulation  output  to  the  corresponding  steady-state  performance  measures  predicted 
by  QBD  approximation  methods.  The  input  of  the  simulated  retrial  queueing  system 
is  a  Markov-modulated  Poisson  arrival  process  with  services  and  breakdowns  that 
are  likewise  modulated  by  the  same  three-state  Markov  chain  whose  infinitesimal 
generator  is  given  by 


-0.2 

0.05 

0.15 

0.3 

-0.4 

0.1 

0.01 

0.7 

-0.71 

(4.37) 


Using  vector  terminology,  we  may  describe  the  exponential  rates  of  the  modulated 
parameters  of  the  simulated  model  as  A  =  [0.3, 0.1,  3],  /i  =  [0.5,0.8,10],  and  £  = 
[0.3, 1.5,  2.1],  The  exponential  rates  ol  of  repair  and  6  of  retrial  were  assigned  the 


same  rates  over  all  environment  states;  that  is,  cq  =  a  =  2.0  and  =  6  =  1.0  for 
each  i  e  S  =  {1,2,3}.  Applying  the  values  of  these  parameters  to  (4.18)  results 
in  a  value  for  the  traffic- intensity  factor  p  of  0.6630,  which  implies  a  stable,  albeit 
relatively  high-traffic  system. 

Table  4.6  displays  the  output  of  the  simulation  runs  versus  the  corresponding 
measures  computed  from  the  steady-state  system  probabilities,  which  were,  in  turn, 
computed  using  the  Bright  and  Taylor  algorithm. 


Table  4.6  Simulation  output  versus  QBD  approximation. 


Performance  Measure 

Simulated 

QBD  Approximation 

E[L] 

7.5678 

7.5784 

E[W] 

11.6288 

11.6346 

E[R } 

7.0551 

7.0645 

E[Wr ] 

10.8402 

10.8456 

P(Busy) 

0.5127 

0.5140 

P(Failure) 

0.2920 

0.2947 

The  results  of  this  chapter  enable  us  to  characterize  the  explicit  stability  con¬ 
ditions  and  steady-state  distribution  of  the  exponential  model  using  the  matrix- 
analytic  method.  In  addition,  we  have  demonstrated  the  utility  of  matrix-geometric 
algorithms  in  approximating  the  steady-state  probabilities,  together  with  numerical 
results  for  the  performance  measures  of  the  system.  To  this  end,  a  simple  criterion 
for  the  ergodicity  of  general  level-dependent  QBDs  was  proved,  thus  contributing 
to  the  understanding  of  the  structure  and  behavior  of  this  class  of  stochastic  mod¬ 
els.  In  the  next  chapter,  a  similar  set  of  results  will  be  established  for  a  model  with 
generally-distributed  service  requirements.  The  analysis  of  the  steady-state  behavior 
of  the  Markov-modulated  M/G/l  retrial  queue  with  unreliable  server  will  have  im¬ 
portant  implications,  not  only  for  the  purpose  of  extending  results  to  non-Markovian 
unreliable  retrial  queues,  but  also  for  elucidating  the  common  elements  that  it  shares 
with  the  Markovian  model. 
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5.  General  Service  Requirements 

This  chapter  extends  the  results  of  Chapter  4  to  investigate  an  unreliable  retrial 
queue  in  a  random  environment  wherein  customers  bring  generally  distributed  ser¬ 
vice  requirements.  Following  a  brief  model  description,  we  explicitly  derive  the 
semi-Markov  kernel  corresponding  to  the  Markov  renewal  sequence  of  orbit  size  and 
environmental  state  considered  just  after  the  system  returns  to  an  up-and-idle  state. 
This  will  subsequently  allow  us  to  state  specific  conditions  for  the  ergodicity  of  the 
embedded  Markov  chain.  The  matrix-analytic  procedure  that  we  employ  makes  sig¬ 
nificant  use  of  the  theory  of  Markov-modulated  Poisson  processes  (MMPPs)  whose 
rudimentary  concepts  are  summarized  in  Section  A. 3  of  the  Appendix. 

5.1  Model  Description  and  Notation 

We  define  the  retrial  queueing  system  in  a  very  similar  manner  to  that  of  the 
Markovian  model,  with  the  primary  difference  being  the  way  in  which  the  service¬ 
time  distribution  is  characterized.  We  again  define  the  continuous-time,  trivariate- 
process  description 

{(R(t),Z(t),X(t)):t>  0}, 

where  R(t )  G  Z+  denotes  the  orbit  size  at  time  t,  Z(t)  G  S  —  {1,2, . . . ,  m)  denotes 
the  environmental  state,  and  X(t)  the  server  status  (0  =  failed,  1  =  idle,  and  2  = 
busy).  The  arrival,  retrial,  failure,  and  repair  processes  are  modulated  by  the  Markov 
chain  {Z{t)  :  t  >  0}  with  environment-dependent  rates  again  contained  in  the  vectors 
A,  6.  £,  and  «,  respectively.  The  environment  modulates  the  exponential  rate 
parameters  of  the  input  streams  via  the  evolution  of  its  own  state  transitions.  In 
other  words,  if  the  environment  is  in  state  k  G  S,  then  the  rate  parameters  are, 
respectively,  A k,  6 k,  6 c,  and  ak. 
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Customers  bring  a  service  requirement  represented  by  a  nonegative  random 
variable  with  a  continuous  distribution  function  that  is  nondefective  and  proper.1 
Define  the  c.d.f.  of  service  time  as  H{t)  with  finite  mean  1  / /jl  and  denote  its  Laplace- 
Stieltjes  transform  (LST)  by 

/»oo 

H(s)  =  /  e~stdH(t). 

Jo 

We  shall  assume  that  the  service  is  not  modulated  by  the  random  environment, 
although  this  assumption  can  be  relaxed  for  the  case  in  which  service  times  are 
distributed  as  PH-random  variables  (see  Appendix,  Section  A. 2.1). 

Customers  arrive  according  to  a  modulated  Poisson  process,  are  serviced  by  a 
single  server,  and  then  leave  the  system.  Failures  likewise  occur  as  a  Poisson  process, 
upon  which  the  server  is  occupied  for  the  entire  length  of  an  exponentially-distributed 
repair  period.  A  customer  already  in  service  when  a  failure  occurs  immediately 
proceeds  to  the  retrial  orbit,  from  whence  it  attempts  to  reaccess  the  server  at 
random  (exponential)  intervals.  Customers  who  arrive  to  a  failed  or  busy  server 
likewise  join  the  retrial  orbit.  The  retrials  themselves  are  conducted  independently 
of  all  others  and  succeed  only  when  the  server  is  up  and  idle.  Departures  from  the 
system,  however,  only  occur  when  a  customer  is  not  interrupted  by  a  failure  during 
the  entire  course  of  its  service. 

A  major  difference  in  the  analysis  of  this  system  comes  about  as  a  result  of  the 
Markov-modulated  input  streams  (otherwise  known  as  Markov-modulated  Poisson 
processes,  or  MMPPs;  see  Appendix,  Section  A. 3).  While  these  also  pertain  to  the 
QBD  {(R(t),  Z(t),X(t))  :  t  >  0},  they  are  not  directly  considered  by  QBD  methods 
since  they  are  implicit  in  the  QBD  structure  of  the  generator  (transition  probability) 
matrix  representation  of  the  system.  Nevertheless,  the  MMPPs  become  a  greater 

1A  probability  distribution  is  nondefective  if  lim^^oo  F(x)  =  1  for  a  (in  this  case,  continuous) 
c.d.f  F(-).  The  c.d.f  is  deemed  proper  if  /0°°  dF{x)  =  1. 
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focus  of  emphasis  as  we  must  now  directly  account  for  their  role  in  determining  the 
stability  conditions  of  the  queueing  system. 


5.2  Derivation  of  the  Semi-Markov  Kernel 

Define  Rn  =  R(Tf),  Zn  =  Z(Tf),  and  Xn  =  X(Tf)  at  the  instants  Tf,  n  = 
0, 1, . . . ,  cx)  just  after  the  server  returns  to  np-and-idle  status.  The  resulting  discrete¬ 
time  process 

{((-Rn,  Zn,  Xn),  Tn)  :  n  >  0}  (5.1) 

is  then  a  Markov  renewal  process  with  kernel  Q*(x)  given  by 

B0(x)  Bi(x)  B2(x)  B3(x)  Ba(x) 

Aq1^)  A^\x)  A^\x)  ^^(x)  A^\x) 

0  A{02\x)  Af'\x)  A^\x)  Af\x) 

0  0  Aq3\x)  Aa3\x)  Af\x) 

0  0  0  A{q\x)  A^\x) 

and  whose  elements,  for  i  >  1,  take  the  form 

=  -R{-Rn+1  =i  +  V  -  1,  Zn+ 1  =  j\  Tn+ 1  <  X 

|  Rn  i ,  Zin—  i  j  | 


and 

[Bu(x)\jjr  =  P{Rn+i  =  u,  Zn+ 1  =  j',  rn+ 1  <  x  I  Rn  =  0,  Zn  =  j},  (5.3) 

given  v,  n  G  Z+,  x  €  M+,  and  i  >  1  (see  [40:  p  157]).  Thus,  for  any  regenerative  cycle 
(Tn,Tn+i\  and  for  v  >  0,  the  matrices  A„\x)  contain  the  probabilities  that  the  orbit 
size  has  increased  by  exactly  v  —  1  if  the  initial  orbit  size  is  not  zero.  The  matrix 
Bu(x)  contains  the  probabilities  that  v  customers  arrive  during  a  regenerative  cycle 
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that  begins  with  an  empty  orbit.  Because  of  the  fact  that  the  regenerative  epochs 
are  the  up-and-idle  instants,  Xn  =  1  for  all  n  >  0,  and  so  we  will  refer  to  the  process 
(5.1)  in  the  abbreviated  form 


{((Rn,  Zn),Tn)  :  n  >  0}. 

It  is  thus  implicit  within  this  expression  that  Xn  =  1  for  all  n  >  0. 

We  henceforth  will  refer  to  the  regenerative  cycles  as  cycles  for  the  sake  of 
simplicity  and  define 

rn  =  Tn+ 1  -  Tn,  n  >  0. 

In  order  to  determine  probabilities  (5.2)  and  (5.3),  we  must  first  enumerate  the 
possible  events  that  may  occur  during  a  cycle.  We  enumerate  these  events  in  a 
similar  manner  to  that  which  was  done  in  [62],  When  one  takes  into  account  the 
fact  that  the  repair  distribution  does  not  vary  based  upon  whether  the  failure  was 
idle  or  active,  we  are  left  with  the  following  scenarios: 

1.  The  server  fails  before  the  arrival  of  a  primary  customer  or  a  retrial 
customer  (if  the  orbit  is  nonempty). 

2.  A  primary  customer  arrives  before  a  failure  or  a  retrial  customer,  but 
the  service  is  interrupted  by  a  failure. 

3.  A  retrial  occurs  before  a  failure  or  a  primary  customer  arrival,  but  the 
service  is  interrupted  by  a  failure. 

4.  The  service  of  a  primary  customer  is  non-interrupted  by  failure  and  is 
completed. 

5.  The  service  of  a  retrial  customer  is  non-interrupted  by  failure  and  is 
completed. 
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Define  In  G  {1, . . . ,  5},  n  G  Z+  as  the  first  event  that  occurs  during  the  cycle 
(Tn,Tn+i],  We  define  the  conditional  probabilities 


P  {Rn-\-l  i  T  V  I;  C  l\  ■  1  j  i 


l^(k,x)}jf  =  ( 


7~n-\- 1  ^  %  |  j)  In  Pn  ^ } 

P  {Pn+ 1  —  ^5  ^n+l  J  j  Ai+1  —  | 


J,  In  /c,  -Rn  0} 


if  i  >  0, 


if  i  =  0, 


and 

PfcW  =  P{In  =  k  |  Rn  =  i}  ,  i  G  Z+,  k  =  1, . . . ,  5.  (5.4) 

Since  the  events  {1, 2, . . . ,  5}  are  mutually  exclusive,  we  may  utilize  the  law  of  total 
probability  to  assert  that 


=  £><?(*,  *)ff,  (5.5) 

k= 1 

5 

£,(*)  =  (5.6) 

fc=i 


It  thus  becomes  clear  that,  rather  than  dealing  with  scalar  values,  each  A„\x)  and 
Bv(x)  for  %  >  0  are  now  block  matrices  of  dimension  m  x  m  (where  m  denotes  the 
number  of  states  of  the  Markovian  environment). 

Our  next  step  is  to  determine  the  probabilities  pff* ,  i  =  0,1,2,...,  k  — 
1,...,5.  Let  Fv(x)  denote  the  c.d.f.  corresponding  to  a  random  variable  Xv  ~ 
PH(cj)TnTv),  where  Tv  =  Q  —  A (77)  (see  Appendix,  Section  A. 2).  We  now  proceed 
to  derive  (5.4)  explicitly  for  the  five  events  that  were  defined  earlier. 


Case  In  =  1:  The  server  fails  before  the  arrival  of  a  primary  customer  or  a  retrial 
customer  (if  the  orbit  is  nonempty). 
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Figure  5.1  Graphical  depiction  of  an  idle  failure  (Case  1). 


For  this  case,  we  assume  the  mutual  independence  of  the  arrival,  uptime,  and  retrial 
processes.  It  is  thus  necessary  to  evaluate  the  probability 

pP  =  P  {Xi  <  v„x}  p  {xi  <  x*} ,  i  e  z+, 

where  X |  denotes  the  random  uptime  of  the  server,  X*  gives  the  interarrival  dura¬ 
tion,  and  X1®  is  the  period  of  time  between  retrials,  given  that  there  are  i  customers 
in  the  retrial  orbit,  each  of  which  is  measured  in  the  interval  (Tn,  Tn+i].  Since 
Xi  ~  T^),  X*  ~  PH((f>x,T\),  and  X*  ~  PH(<t>id,Tie),  (1.15)  gives  ns 

P{  Xi<X$  =  l  +  (4>t®4>x)(-Tt®Tx)-\e®Txe)  (5.7) 

P{Xi<Xlne}  =  l  +  (<i>t®<t>i0){-Tt<BTio)-1(e®Tiee).  (5.8) 

Multiplying  (5.7)  and  (5.8)  gives, 

pf  =  1  +  ©  0A)(-r?  ©  rA)-1(e  ©  Txe),  (5.9) 

p?  =  Pi(0)[l  +  (^  ®  Mi-Tz  ©  Tie)-\e  ©  Tiee)l  i  >  1.  (5.10) 

For  arbitrary  MMPP  rate  parameters  a  and  b ,  define 

Po(a,b)  =  P{Xan<Xbn }  =  1  +  (<t>a  ©  <t>b)(—Ta  ©  Tft)_1(e  ©  Tbe). 
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Figure  5.2  Graphical  depiction  of  a  failure  that  occurs  during  the 
service  of  a  (primary)  customer. 


It  is  clear  from  the  preceding  definition  that 

Po(a,b)  =  1  -  Po(b,  a). 

When  applied  to  equations  (5.9)  and  (5.10),  we  obtain,  respectively, 

b0)  =  a) 

p<‘>  =  pWp0(£,,e). 

Case  In  =  2:  A  primary  customer  arrives  before  a  failure  or  a  retrial  customer,  but 
the  service  is  interrupted  by  a  failure. 

This  scenario  differs  from  Case  1  in  that  we  must  compute  the  probability  of  a 
failure  occurring  before  the  end  of  a  service  period,  which  is  arbitrarily  distributed. 
Denote  the  random  variable  X£  to  be  the  total  service  time  requested  during  a  cycle 
(Tn,Tn+i\  of  Type  2.  We  must  state  the  probability  of  a  failure  occurring  before  the 
end  of  a  service  period  in  the  following  manner: 

P  {Vi  <  X*  +  Vi}  =  P  {Vi  -  Vi  <  Vi}  .  (5.11) 

Denote  the  difference  X^  —  Xx  by  Xf~x.  We  now  seek  to  determine  the  distribution 
function  Fc  \  associated  to  the  random  variable  X|_A.  To  this  end,  we  condition  on 
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the  arrival  time  of  the  primary  customer  and  note,  as  in  Figure  5.2,  that  if  we  are 
given  arrival  time  s  >  0,  then  for  any  a  >  0,  <  a  if  and  only  if  it  is  no  larger 

than  the  interval  [s,  s  +  a] .  This  is  given  by  the  conditional  probability 

P  {Xf  <  a  I  XA  <  Xi}  . 


We  must  therefore  compute  the  joint  probability 


P{XtX  <  a ,  A'A  <  Xii 


dF^(t)  dF\(s ) 


=  [0$(exp (T^a)  -  I)  ©  (f)x]  (-T5  ©  Tx)  1  (e  ©  Txe) , 


from  which  we  obtain 


F(A a)  =  P{XtX  <  a  I  Aa  <  X«} 

P{XTx  <  a ,  XZ  <  Xj) 

P{Xi  <  Xk] 

=  [^s(exP(r.a)  -1)0  <h]  (~T(  efitM  Ue)  . 

1  +  {4>x  ©  4>t:)[-Tx  ©  m~1(e  ©  T?e) 

We  next  proceed  to  compute  the  probabilities  of  occurrence  for  Case  2.  Once 
again,  invoking  the  mutual  independence  of  the  various  MMPPs  allows  us  to  state, 
for  i  >  0,  that 

ff  =  P{In  =  2  R„  =  i]  =  P{ A'A  <  A«}  P{Xk  <  X'ne }  P{ Xtx  <  X"}.  (5.13) 


For  a  square  matrix  M,  define  the  integral  operator 


A  (M)  =  /  exp  (—Mt)dH(t), 
Jo 


where  H(-)  is  the  c.d.f.  of  service  time.  Using  this  notation,  we  expand  (5.13)  to 
obtain 


P?  =  Po{\,Z)Po{\iO)  I  Ft,x(u)dH{u) 

Jo 

=  P0( A,  id)  ^  J  (exp (T(u)  -  /)  dH(u)  ©  (j)^J  (-Tc  ©  TA)_1  (e  ©  Txe) 
=  P0(A,  id)  (^(A(-T?)  -  /)  ©  0A)  (-Tc  ©  Ta)"1  (e  ©  TAe).  (5.14) 

Expression  (5.14)  may  now  be  explicitly  evaluated  if  the  distribution  function  H(-) 
of  the  service  requirements  is  specified. 


Case  In  =  3:  A  retrial  occurs  before  a  failure  or  a  primary  customer  arrival,  but  the 
service  is  interrupted  by  a  failure. 

This  situation  is  very  similar  to  that  of  Case  2,  except  that  the  arrival  is  a  retrial 
customer.  If  we  define  the  random  variable  X^~10  in  an  analogous  manner  to  X^~x, 
then,  for  i  >  1, 

P^  =  P{In  =  3\Rn  =  i} 

=  P  { X *  <  X*}  P  {X*  <Xxn}P  { xtie  <  Xn} 

=  P0(iG ,  A)  [4>c  (A(-T?)  -  I)  ©  (f)ig\  (-Ts  ©  Tie)-1  (e  ©  Tiee). 

where  F^ig{u)  is  the  c.d.f.  of  the  random  variable  corresponding  to  the  difference 

-  x« 

Case  In  =  4:  The  service  of  a  primary  customer  is  non-interrupted  by  failure  and  is 
completed. 
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Assume  now  that  the  end  of  service  occurs  before  a  failure.  This  may  be  expressed 
for  i  >  0  as 


p f  =  p0(\,t)P0(\iO)(i-p{xtx<xz}) 

=  P0( A,  iff)  ^P0(A,  £)  -  ^  (exp (T(U)  -  I)  dH(u)  0 

x  (T,  ©  Ta)-1  (e  ®  TAe) 

=  Pq(\,  iff)  [Po(A,O-(^(A(-rC)-/)®0A)(-^©TA)-1(e®TAe)]. 


Case  In  —  5:  The  service  of  a  retrial  customer  runs  to  completion. 
This  is  the  retrial  version  of  Case  4.  Accordingly,  for  i  >  1,  we  obtain 


p5(i)  =  p0(id,t)p0(ie,\)[i~p{xtie  <x%}] 

=  P0(iff,  A)  (^Po(iO,  £)  -  J  (exp (T$it)  -  /)  dP(u)  0 
x  (T.c  ©  (e  0  Tiee) 

=  P0(iff,  A)  [Po(i0, 0  -  (^(A(-Ts)  -7)0  0* )  (-T*  ©  T,,)-1  (e  0  Ti0e)\  . 


In  much  the  same  case-by-case  fashion,  we  will  derive  expressions  for  the  gen¬ 
erating  functions  A^*(z)  and  B*(z),  i  >  0,  \z\  <  1,  for  the  block  expressions  of 

the  semi-Markov  kernel  G(x).  Let  us  define  the  Laplace-Stieltjes  transform  (LST) 

matrices 

POO 

V®(k,s)  =  /  e~sxd¥j\k,x),  (5.15) 

J  o 

and  their  ^-transforms  with  respect  to  v  as 

OO 

¥^*(k,s,z)  =  ^2^\k,s)zu.  (5.16) 

i/=0 
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For  notational  convenience,  kernel  expressions  that  do  not  depend  upon  the  current 
number  in  orbit  do  not  carry  the  superscript  i.  In  such  cases,  we  write 


^\k,x)  =  ^fv(k,x) 

¥^(k,s,z)  =  V*(k,s,z). 

Define  the  Laplace-Sticltjes  transforms  of  the  kernel  elements  Av\x)  and  Bv{x)  of 
the  Markov  renewal  process  {((i?n,  Zn),  Tn), :  n  >  0}  as 


The  entries  of  the  transition  probability  matrix  of  the  Markov  chain  { ( Rn ,  Zn)  :  n  > 
0}  may  be  obtained  by  taking  the  limit  of  the  kernel  entries  as  x  — >  oo  such  that 


= 

V 

lim  A^\x) 
>00 

=  hmAW(s) 

s^O 

(5.17) 

Bv  = 

lim  Bu(x) 

x^O 

=  lim  Bv{s). 

s— >0 

(5.18) 

Define  the  ^-transforms 

OO 

A^’(s,z)  =  £>MW(») 

i/=0 

oo 

B*(s,z)  =  J2zvBv(s). 

v=0 


101 


We  then  form  the  stochastic  matrices  and  B  as  follows: 


A' (i)  =  =  -4W*(0,1),  (5.19) 

u=0 

OO 

B  =  (5.20) 

v= 0 


Now  let  <&v(v,t)  be  the  MMPP  counting  process  defined  in  (1.11)  of  the  Ap¬ 
pendix.  An  important  fact  that  we  will  frequently  use  is  the  following,  which  appears 
in  [40]. 

Theorem  5.1.  Let  rj  be  the  vector  containing  the  exponential  rate  parameters  of  a 
MMPP  { Z(t )  :  t  >  0}  with  generator  Q.  Then 

°°  f  POO  N  POO 

Y  \  /  $r,{v,t)dFr,{t)\zv  =  /  exp((Q-  (1  -  z)A(r]))t)dFn(t).  (5.21) 

u=o  J  Jo 

Proof.  The  proof  centers  around  the  ability  to  interchange  the  integral  and  sum, 
which  is  possible  if  and  only  if  the  function 

OO 

£  t)  (5.22) 

v= 0 

is  uniformly  convergent  with  respect  to  v  —  0, 1,  2, . . ..  Consider  the  Poisson  proba¬ 
bility  that  n  arrivals  occur  in  the  interval  (0,  £],  which  is  given  by 

P(n,t)  =  e~vt(^~,  n  >  0,t  >  0.  (5.23) 

ni 


If  we  let  the  rate  r/  vary  so  that  P(n,  t )  =  P(n,  t,  rj)  and  take  its  derivative  with 
respect  to  g,  we  obtain 


d_ 

di i 


P(n,t,v) 


(n  —  1)! 
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It  is  clear  that  the  derivative  will  be  negative  for  any  rj  >  0  if  we  are  given  large 
enough  t.  In  other  words,  P(n,t,r)i )  will  be  decreasing  for  each  %  =  1 .m  given 
t  >  t*  for  some  t*  >  0.  Furthermore,  if  r/,  >  rjj,  then  P(n,t,rji )  <  P(n,t,r}j),  which 
then  proves  that 

P(n,  t,  r/min)  >  <h(n,  t),  i  =  (5.24) 

where  r/m;n  =  min We  also  point  to  the  fact  that 


lim  P(n,  t,  77min)  =  0.  (5.25) 

» oo 


Due  to  the  verification  of  conditions  (5.24)  and  (5.25),  we  may  now  invoke  a  well- 
known  theorem  of  Weierstrass  in  which  these  conditions  are  necessary  and  sufficient 
for  the  uniform  convergence  of  expression  (5.22).  Consequently,  we  may  now  inter¬ 
change  the  operations  of  summation  and  integration  in  (5.21)  to  obtain 

00  C  poo  ''j 

£  /  ^(v,t)dFn(t)  \z" 

u= 0  Wo  J 


u=0 


Y,  (  /  P{N{t)  =  V,  Z(t)  =  f  I  N{ 0)  =  0,  Z{ 0)  =  j}  dF.it )  )  z 


^  P{N(t)  =  v,  Z(t)  =  j'  |  N (0)  =  0,  Z( 0)  =  j}  zZ  )  d,F.{t) 

\  u=0 

exp  HQ  -  (1  -  z)A(rf))t)  dF.it), 


where  the  final  equality  follows  from  Lemma  A. 2  in  the  Appendix.  □ 

Since  the  only  entity  count  that  we  require  from  any  MMPP  is  that  of  customers 
arriving  to  the  queue,  we  use  the  shorthand  notation  <h(z/,  t)  =  <LA(z/,  t)  (see  (1.11) 
in  the  Appendix)  for  the  remainder  of  the  chapter.  In  addition,  define 


T\(z)  =  Q  -  (1  -  z)A(\) , 
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which  shall  serve  as  a  convenient  shorthand  in  the  complicated  derivations  upon 
which  we  shall  now  embark. 

It  often  becomes  necessary  during  the  computation  of  the  semi-Markov  ker¬ 
nel  to  count  the  number  of  arrivals  during  repair  periods  and  generally-distributed 
service  times.  In  order  to  facilitate  this  procedure,  we  define  the  matrices 


J?(x 

)  =  [  ^(v,t)dFv(t), 

Jo 

(5.26) 

)  =  I  $(v,t)dFttX(t), 

Jo 

(5.27) 

Ju(X, 

)  =  [X${v,t)dH{t), 

Jo 

(5.28) 

where  r]  is  the  vector  of  rate  parameters  of  a  MMPP.  We  will  likewise  find  it  necessary 

to  utilize  the  LST  JrJ(s)  of  J„(x)  and  the  corresponding  ^-transform  matrix  s ), 

which  are  defined  as 


J?As)  = 


’dJ'J(x) 


(5.29) 


Jo 

OO  />oo 

J"*{S,Z)  =  y V/  e~sx  dJ^x).  (5.30) 

i/=0 

The  following  theorem,  which  is  a  restatement  of  [82:  Thm  5.1.5],  allows  the  explicit 
computation  of  (5.29)  evaluated  at  s  =  0: 

Theorem  5.2.  The  matrix 


roc 

JM  =  /  *(v,x)dFv(x) 
Jo 

is  given  by 


J0V( 0)  =  (I  <8>  (f>v)(T\  ©  Tr/)_1(7  ®  Tve)  (5.31) 
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J2(  0)  =  DCU~1E, 


v  >  1 


where  the  matrices  D,  C ,  and  E  are  given  by 


C  =  (0A®/)(TA©T,)-1(TAe®  I) 

e  =  (0A®/)(rA©r„)-1(/®^e). 


The  transform  matrix  (5.30)  may,  in  turn,  be  evaluated  at  s  —  0  and  z  —  1  using 
Theorem  5.1: 


1)  =  (/  ®  4>v)  z"  /  $(z/,  x)  ®  exp(T„a;)  dx{I  ®  T„e) 

v=0  ^ 

/»00  00 

4>v)  /  Y''  ^^(z/,  x)  ®  exp(Tr?x)  dx(I  ®  T^e 

^=o 

/*oo 

4>v)  /  exp(Q  —  (1  —  z)A(A)  x)  ®  exp(Tr?x)  dx{I  ®  Tve) 


z= 1 


z= 1 


z= 1 


(/ 

(/ 


4>n)  /  exp  ([Q  ©  Tjjjx)  dx(I  ®  Tne) 


<l>v)  (~Q  ©  Tv)  1  (/  ®  T,ne). 


We  will  now  complete  the  derivation  of  the  semi-Markov  kernel  block  entries  A\^ 
and  Bv  as  defined  in  (5.17). 


Case  In  =  1:  The  server  fails  before  the  arrival  of  a  primary  customer  or  a  retrial 
customer  (if  the  orbit  is  nonempty). 
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In  this  scenario  the  up-idle  period  ends  with  a  failure,  which  is  then  followed  by  a 
repair.  No  customers  are  processed  during  this  cycle,  and  thus 

'b0(2,  x)  —  0  for  x  >  0. 


For  v  >  1,  the  cycle  commences  with  an  up-idle  period  followed  by  a  failure,  and 
then  a  repair  period  during  which  primary  arrivals  may  occur.  Hence,  we  employ 
the  renewal  argument  by  conditioning  upon  the  time  of  failure  as  follows: 


'Ml.i) 


J“-i(x-t)dFt{t), 

Jo 


x—t 


*( 


V 


1,  u )  dFa{u)  dF^(t), 


(5.32) 


where  JJ(x)  is  the  matrix  defined  in  (5.26).  Since  we  require  the  derivative  of  (5.32), 
it  is  necessary  to  reduce  this  expression  to  one  that  requires  only  a  single  integration. 
We  thus  employ  the  technique  of  partial  integration  to  obtain  the  equivalence 


dV(l,o;)  =  -  (/®0£  <g)  <f)a)  J  (<£>(*'-  M)  <8>exp(((-T?)  ®Ta)t))dt 

x  (7<g>  exp(T^a:)e  <g)  Tae ). 

(5.33) 


We  next  turn  our  attention  to  the  evaluation  of  the  LST  of  ^(1,2;),  \&„(l,s),. 
which  may  be  obtained  as  follows: 

r  oo 

$,(M)  =  /  e-“#,(l,i) 


=  J“_i(s)  -  (/  <8>  0^  ®  0a)<  /  (e  sx$(v  -  l,x)  ®  I  ®  exp(Tax)Tae)  dx 


r*00  pX 

—sx 


+  I  e  ax  f  <F(z/,  t)  ®  exp(((— T^)  ©  Ta)t)  dt 
'o  Jo 
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x  (7(8)  exp(T^x)T^e  ©  Tae )  efe 


(5.34) 


from  which  we  may  compute  the  conditional  probabilities  ^©(l,  oo)  of  the  occurrence 
of  v  —  1  arrivals  to  the  retrial  orbit  in  a  cycle  in  which  In  —  1: 


\Eh,(l,cx>)  =  lim  '&v(l,x)  =  \E©(l,s) 


=  J?-M- 


(5.35) 


s=0 


It  is  possible  to  obtain  a  closed-form  solution  of  (5.35)  through  the  application  of 
Theorem  5.2.  In  this  way,  we  determine  that 


f(/®  (f>a)(Tx®Ta)  l(I®Tae)  if  v  =  0 
T„(l,oo)  =  J“(0)  =  <  ,  (5.36) 

[dc'-'e  a  u  >i 

where  the  matrices  D,  C,  and  E  are  given  by 


D  =  (. I®<j>a){Tx@Ta)-\Txe®I ) 
C  =  (0A®/)(TA©Ta)-1(TAe®/) 
E  =  ((j)x  ®  I)(T\  ©  Ta)_1(/  0  Tae). 


Our  final  task  is  to  derive  the  generating  function 

OO 

^{i,s,z)  =  ^u(i,s)zv. 

u=0 
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From  (5.34)  we  thus  obtain 


**(1,0,*)  =  z°-  o  + 

v=\ 

=  zJ"*(0,  s,  z) 

=  {I  ®^a)z{-Tx{z)®Tayl  (I  ®Tae). 


Evaluating  z  at  1  gives 


(I  ®(t)a)(-Q®Ta)  1  (I  ®Tae).  (5.37) 

The  quantity  (5.37),  as  well  as  its  counterparts  0, 1),  k  =  2, . . . ,  5,  will  be  re¬ 
quired  to  derive  the  stochastic  matrices  (5.19)  and  (5.20). 

Case  In  =  2:  A  primary  customer  arrives  before  a  failure  or  a  retrial  customer,  but 
the  service  is  interrupted  by  a  failure. 

We  count  arrivals  during  both  of  an  interrupted  service  epoch  and  the  repair  period 
that  follows.  Because  the  customer  in  service  will  inevitably  join  the  retrial  orbit, 
the  definition  of  4/^(2,  x)  implies  that 

4^(2,  x)  —  0  for  v  =  0, 1. 

For  v  >  2,  it  is  necessary  to  consider  arrivals  over  both  of  a  truncated  service  and 
repair  epochs,  the  sum  total  of  which  must  equal  v  —  2.  Let  JW~a+q  denote  the 
random  variable  for  the  interval  of  time  during  which  such  an  interrupted  service 
and  repair  takes  place.  As  in  Case  1,  we  condition  upon  the  arrival  time  t  of  the 
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first  customer  arrival  to  obtain,  for  t  G  (0,  x)  and  v  >  2,  the  expression 


=  /  J^(x~t)dFx(t) 


(5.38) 


a?  px—t 


<f>(z/  -  2,  u)  dF^a(u)  dF\(t), 


o 


where  we  denote  the  c.d.f.  of  the  distribution  of  the  combined  interrupted  service 
and  repair  intervals  by  F^x,a{t)  and  define 

J^x,a(X)  =  [  $(©  u)  dF^  X  a{u). 

Jo 

In  an  analogous  manner  to  Case  1,  we  arrive  at  the  following  expression  for  the 
probability  of  the  orbit  size  incrementing  by  v  —  1  during  a  cycle  in  which  In  =  2: 

/»oo 

^,(2,00)  =  tf„(0)  =  f(0)  =  /  $(z/-2  ,x)dF^a(x). 

Jo 

In  order  to  evaluate  this  integral  any  further,  we  must  determine  the  c.d.f.  F^^ait) 
of  the  sum  X^_A+a  =  XFX  +  Xa.  This  can  be  computed  in  the  following  manner: 

F(,\AX)  =  /  Fa[x  ~  tydF^At) 

Jo 

=  F^x(x)  -  (0a  ®  ®  0Q)  [  exp (((-Tq)  ©  Tt)t)  ®Idt 

Jo 

x  [exp(Tax)e  ©  (-Xg  ©  TA)_1(e  ©  Txe)\ 


=  F^x(x)  -  (0a©0£©0a) 

x  |  [(exp(((— Ta)  ©  T^)x)  -/©/)]  ©  / 


x  [exp(Tax)e  ©  (—  ©  Tx)  1(e  ©  Txe)\  . 


(5.39) 
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From  here,  we  proceed  directly  to  the  computation  of  the  ^-transform  \Ef(*)*(2,  0,  z) 
evaluated  at  z  —  1.  As  in  Case  1,  we  begin  with  the  definition  of  the  ^-transform  to 
obtain 


d'('*)*(2,  0,  z) 


0  •  z°  +  0  ■  z  + 

V=1 


z2JiXoi\$,z) 


=  |(/  (8)  0<  ©  0a)  b2(~Tx(z)  ©  T^)-1  ©/][/©  fa  ©  /)]  }/P0(A,  £) 
-  (/  ©  4>a  ©  05  ©  0J  |  [;s2(-TA(z)  ©  /  ©  Ft)-1  ©  I] 
x  {(/  ©  (— Ta)e  ©(/©/))  +  (/©  e  ©  (Tc  ©  /))  +  (/  ©  T„e  ©  (/  ©  /))} 


'z2(-Tx(z)  ©  Tq)_1  ©  (/  ©  /)]  (/  ©  Tae  ©  (/  ©  /)) 


-P 


>FAe). 


Case  In  =  3:  A  retrial  occurs  before  a  failure  or  a  primary  customer  arrival,  but  the 
service  is  interrupted  by  a  failure. 

We  now  assume  that  a  failure  occurs  during  the  service  of  a  retrial  customer,  and  thus 
the  current  orbit  size  %  does  not  increment  by  one  at  the  failure  epoch.  Therefore, 
\E'o'>(3,  x)  =  0  and,  for  v  >  1, 

^}(3,x)  =  [  jlte{a{x-t)dFld{t)  (5.40) 

Jo 

x—t 

<f>(z/  -  1  ,u)  dF^i0ta(u)  dFid(t), 


where  we  denote  the  c.cl.f.  of  the  distribution  of  the  interrupted  service  and  repair 
interval  by  and  define 

Jl'*'a{x)  =  [  $(u,u)dFsiie,a(u). 

Jo 
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With  this  established,  it  is  a  simple  matter  to  adapt  the  results  of  Case  2  in  making 
following  statements,  the  first  being 


^,(3,oo)  =  ^(3,0)  =  Jj3’“(0)  =  /  $(v-l  ,x)dFwta(x), 

Jo 

where  the  c.d.f.  F^i$ja(x)  of  the  interrupted  service  and  repair  distribution  is  given 
by 

rx 

F^i9,a(X)  =  /  Fa{x  -  t)dF^ie{t) 

Jo 

=  F^ie (x)  -  {(f) a  ©  ®  <f>a) 


x  <  [(exp(((— Ta)  ©  T^)x)  -  I  0  /)]  ©  I 


x  [exp(Tax)e  ©  (— Xg  ©  Tie)  1(e  ©  Tige)]  .  (5.41) 


The  ^-transform  expressions  \IW)*(3,  0,  z)  may  likewise  be  computed  as 


4/^*  (3,  0,  z)  = 


o-^+E^wo) 


u=l 


=  zJt’i6’a*{U,z) 


=  \{I  ©  ©  (f)l6)  \z{-Tie{z)  ©  T^)-1  ©/][/©  (T?  ©  /)]  \/P0(iO,Z) 


{I  ©  (f>a  ©  ©  <t>a)  [*(-T*(z)  ©  /  ©  Tg)-1  ©  /] 


x  {(/  ©  (— Ta)e  ©(/©/))  +  (/©  e  ©  (T?  ©  /))  +  (/  ©  TQe  ©  (/  ©  /))} 

-  [z(-T*  (z)  ©  Tq)_1  ©  (/  ©  /)]  (/  ©  Tae  ©  (/  ©  /))  j  (-7>  ©  Tie)~l{e  ©  Tiee). 


Case  In  =  4:  The  service  of  a  primary  customer  is  non-interrupted  by  failure  and  is 
completed. 


Ill 


Conditioning  on  the  first  arrival  time,  we  obtain  the  following  for  v  >  1,  where  Jx 
is  dehned  as  in  (5.28): 


H( 

V  ' 


X  px—t 


*„(4,x)  =  /  =  /  /  $(1/  -  1,  u)  dH(u)dFx(t).  (5.42) 

Jo  Jo  Jo 


As  before,  we  use  partial  integration  to  establish  the  equivalence 


X  px—t 


<f>(z/  —  1,  u)  dH{u)dF\(t ) 


o  ./o 


=  /  <f>(z/ —  1  ,x  —  t)F\(t)dH(x  —  t)  =  /  <h(z/ —  1  ,u)Fx(x  ~  u)  dH(u) 


—  J„-i(x)  —  (/  ®  0a)  J  —  l,u)  ®exp(—T\u)  dH(u)  (I  ®exp(T\x)). 


The  LST  of  this  expression  may  be  computed  as 


■MM) 


=  t«,(0)-(/®<AA) 


s=0 


(/  (8)  exp(TAa;)e  dH(x ) 


r»oo  /»rr 


+  /  /  (<h(z/  -  l,w)  8)  exp(-Tyu))(J  8)  exp(TAa;)rAe)  dif(u) 

Jo  Jo 


from  which  we  obtain 


*,(4,0)  =  7^(0) 


The  ^-transform  T*(4,0,z)  is  thus  given  by 


T*(4, 0,  z)  =  z( 


00  r  /»oo 

•0  +  zj"zl/~1  /  4>( 

,=i  LJo 


(i v  —  1,  x)  dH(x ) 


=  z  exp (T\(z)x)  dH(x) 


=  zA(-Tx(z)), 
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which  gives 


(5.43) 


**(4,0,1)  =  A  H3). 


Case  In  =  5:  The  service  of  a  retrial  customer  is  non- interrupted  by  failure  and  is 
completed. 

This  case  is  the  only  one  of  the  five  in  which  the  orbit  size  may  decrement.  The 
appropriate  integral  expression  for  v  >  0  is  thus  given  by 

f*X  px  px — t 

T«(5,a;)  =  /  J? (x  -  t)dFie(t)  =  /  /  $(i/,  u)  dH(u)dFid(t).  (5.44) 

Jo  Jo  Jo 


From  Case  4,  we  thus  have  that 


M5,0)  =  •/"(()). 

The  corresponding  probability  generating  function  is  therefore  given  by 

^W*(5,0  ,z)  =  A  (~Tx(z)), 
from  whence  we  conclude  that 


T«‘(5,0,l)  =  A(-Q).  (5.45) 

With  the  kernel  of  the  embedded  chain  {(Rn,  Zn)  :  n  >  0}  now  specified, 
we  may  investigate  the  conditions  for  the  ergodicity  of  the  system  discussed  in  this 
chapter.  An  additional  prerequisite  to  the  determination  of  stability  conditions  is 
the  establishment  of  stability  criteria  for  level-dependent  discrete-time  M/G/  1-type 
Markov  chains  using  the  theory  of  ergodicity  for  general  Markov  chains.  This  will 
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enable  us  to  determine  the  exact  criteria  for  the  stability  of  the  embedded  chain, 
whose  recurrence  properties  mirror  that  of  the  process  in  which  it  is  embedded. 

5.3  Stability  Analysis 

In  this  section,  we  shall  present  criteria  for  the  positive  recurrence  (and  hence, 
ergodicity)  of  a  general  level- dependent  M/G/  1-type  Markov  chain  {( Ln,Yn )  :  n  > 
0}.  Following  this,  we  will  derive  an  explicit  stability  formula  for  the  embedded 
process  Q*( oc)  of  the  Markov-modulated  unreliable  M/G/ 1  retrial  queueing  system. 
We  first  provide  the  necessary  definitions,  which  are  the  level-dependent  analogues  to 
those  provided  in  Section  3.2.4.  The  determination  of  stability  criteria  for  a  M/G/ 1- 
type  Markov  chain  revolves  around  the  stochastic  matrix  which  we  previously 
defined  in  (5.19): 

OO 

A(i)  =  £><)>,  (5.46) 

u=0 

where  A^  =  A^  (oo)  and  i  >  0.  In  order  to  apply  what  was  done  for  level- 
independent  discrete-time  M/G/  1-type  Markov  chains  in  Section  3.2.4,  we  define 
the  following  for  i  >  1: 

OO 


P{i)  =  5>-4« 

v=l 

(5.47) 

p(’)  =  7TW/3we, 

(5.48) 

where  we  define  nA  to  be  the  invariant  probability  vector  of  the  stochastic  matrix 
A*A .  Consequently,  7r^  contains  as  entries  the  steady-state  probabilities  of  {Yn  : 
n  >  0},  given  a  starting  orbit  size  of  i  >  1. 

In  order  to  apply  the  ergodicity  results  of  [87]  and  [95],  we  must,  nevertheless, 
determine  the  level-dependent  drift  d{i)  of  the  embedded  chain  {( Ln,Yn )  :  n  > 
0}.  This  may  be  obtained  by  noting  that,  from  definition  (5.5)  of  A„\  where  i  = 
1,  2,  3, . . .  and  v  =  0, 1,  2, . . .,  the  increment  of  the  process  is  not  z/,  but  is  actually 
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v  —  1,  given  that  there  are  i  >  1  in  the  retrial  orbit  at  the  beginning  of  a  cycle.  In 
light  of  this  observation,  we  arrive  at  the  following  Lemma: 

Lemma  5.1.  Suppose  that  the  discrete-time  process  {( Ln,Yn )  :  n  >  0}  is  a  M/G/l- 
type  Markov  chain.  Then  its  drift,  d(i)  is  given  by 

d(i )  =  -  1. 

Proof.  We  proceed  using  the  realizations  v  —  1,  where  v  —  0, 1,  2, . . of  the  random 
increments  of  {Ln  :  n  >  0}  and  the  definition  of  expectation: 

OO 

d{i)  =  ttW  ~  l)^°e 

u= 0 

OO  OO 

=  7T^  —  7T^ 

^=0  ^=0 

=  7TW/3we-7T 

=  p(i)  -  1.  (5.49) 

□ 

Thus,  the  quantity  (5.49)  is  the  drift  at  level  i  for  an  M/G/l-type  Markov  chain.  It 
is  clear  that  p®  <  1  holds  if,  and  only  if,  dfi)  <  0,  and  so  we  designate  as  the 
conditional  traffic  intensity  formula  for  the  embedded  Markov  chain  {( Ln,Yn )  :  n  > 
0}. 

As  with  any  DTMC,  the  recurrence  properties  of  the  M/G/  1-type  process  are 
predicated  on  the  behavior  of  the  drift  (5.49)  across  levels  of  the  process.  We  use 
this  relationship  to  define  a  simple  condition  that  is  equivalent  to  the  ergodicity 
of  an  (aperiodic)  M/G/l-type  DTMC  {(Ln,Yn)  :  n  >  0},  where  Ln  denotes  the 
level  of  the  process.  It  is  clear  from  Pakes’  Lemma  that  the  necessary  condition  for 
positive  recurrence  of  {( Ln,Yn )  :  n  >  0}  is  that  limsup^^  d{i)  <  0,  or,  equivalently, 
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lirnsup^^  pO)  <  1.  We  now  specify  this  condition  for  the  limit,  rather  than  the  limit 
supremum,  in  order  to  demonstrate  a  sufficient  condition  for  positive  recurrence,  ft 
is  once  more  necessary  to  utilize  Theorem  3.7  in  validating  this  criterion,  which,  in 
turn,  requires  that  we  demonstrate  Kaplan’s  condition  (see  Section  3.1.3)  for  the 
process  {(Ln,Yn)}. 

Showing  that  Kaplan’s  condition  holds  is  equivalent  to  proving  that  the  se¬ 
quence  of  real-valued  functions  {'ipi(z)  :  i  G  Z+,  z  G  [0,1)}  (see  (3.25))  is  bounded 
from  below.  This  sequence,  written  explicitly,  becomes 


A(z)  =  \z*^£p®z*\ni-z) 


i/=0 


(7 T®A®e)zi+v-1  \  /(I  -  z) 


v= 0 


=  < 


-  tt«  \  /(I  -  z), 


(5.50) 


v=0 


where  p$  is  the  probability  that  the  size  of  the  orbit  transitions  from  i  to  i  +  v  —  1. 
It  is  necessary  to  show  that  there  exist  numbers  c  G  (0,1),  B  >  0,  and  N  >  0  such 
that  ipi (z)  >  —B  for  z  G  [c,  1)  and  i  >  N.  In  other  words,  there  must  exist  real 
numbers  c  G  (0, 1)  and  N  >  0  such  that 


inf  {^(2)  :  z  e[c,  1)}  >  - B , 


holds  for  every  i  >  N.  Such  a  determination  is  complicated  by  the  presence  of  the 
infinite  sum  in  (5.50),  and  so  a  calculus  approach  to  the  minimization  of  'ij>i(z)  would 
be  a  formidable  task.  We  therefore  choose  an  alternate  route  to  the  verification  of 
Kaplan’s  condition  for  the  M/G/  1-type  DTMC,  one  that  focuses  upon  the  conditions 
listed  in  Theorem  3.8. 
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Theorem  5.3.  An  irreducible,  aperiodic,  and  discrete-time  level- dependent  M/G/l- 
type  Markov  chain  {( Ln ,  Yn)  :  n  >  0}  is  positive  recurrent  if  and  only  if  the  following 
conditions  hold: 

OO 

lim  pW  <  1,  and  V  vBv  <  oo,  (5.51) 

i — >oo 

v=0 

where  p^>  is  defined  as  in  (5.48). 

Proof.  The  outline  of  the  proof  is  generally  the  same  as  that  for  Theorem  4.2,  except 
that  we  now  investigate  the  stability  of  the  Markov  chain  {( Ln,Yn )  :  n  >  0}.  We 
begin  with  the  assumption  that  (1)  p ^  <  1,  and  (2)  UBU  <  oo.  It  is  clear 

from  (5.49)  that  (1)  implies  that  the  drift  d(i)  is  negative  in  the  limit.  In  order  to 
facilitate  the  discussion  of  (2),  we  observe  that 

OO 

d( 0)  =  vBu  <  oo. 

v=0 

This,  combined  with  the  decreasing  nature  of  the  drift  over  levels  of  the  process, 
demonstrates  that  the  remaining  drift  terms  for  i  >  1  are  likewise  finite.  Invoking 
Pakes’  lemma  as  before,  we  thus  obtain  the  positive  recurrence  of  the  Markov  chain. 

We  next  prove  the  reverse  implication,  namely  that  in  which  we  assume  the 
hypothesis  does  not  hold  and  {(Ln,Yn)}  is  ergodic.  We  shall  derive  a  contradiction 
using  [95:  Thm  1],  which  requires  the  fulfillment  of  Kaplan’s  condition  as  an  initial 
step.  As  previously  mentioned,  we  will  avoid  an  explicit  verification  of  this  condition, 
and,  instead,  proceed  to  verify  condition  (2)  of  Theorem  3.8.  This  requires  a  proof 
of  the  existence  of  a  lower  bound  for  the  sequence  {<5*  :  i  =  0, 1,2,.. .},  where 

^ 

j<i 

and  pij  is  the  (i,j)th  element  of  the  transition  probability  matrix  P  of  {(Ln,Yn)}. 
We  refine  the  expression  for  <5*  by  noting  that,  for  the  state  space  S  of  the  process 
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{(Ln,Yn)}, 


mm{jf  -i:i>j,i,jeS}  =  - 1, 

since  the  only  decrement  (i.e.,  transition  to  a  lower  level)  that  a  M/G/  1-type  process 
is  allowed  is  -1.  Applying  this  observation  leads  to  the  following  for  i  >  1: 

A  =  ^Pij(j-i) 

j<i 

=  (— 7r(i_1)Ao  _1)  +  0  •  7TWA?)e 

=  -7 T^At1]e 

>  -1 

with  (5.52)  a  result  of  the  fact  that  n is  the  unconditional  probability  of  an 
increment  of  v  —  1  in  the  level  for  z/  >  0.  Hence,  we  have  shown  that  Kaplan’s 
condition  holds. 

Finally,  we  recall  that  from  Lemma  5.1  that  the  drift  at  level  i  is  given  by 
d{i )  =  pb)  _  i  Since  we  assume  that  lim^oopb)  >  there  must  exist  some  >  0 
such  that  d(i)  >  0  whenever  i  >  N.  By  Theorem  3.7,  we  conclude  that  the  M/G/ 1- 
type  DTMC  {(Ln,Yn)}  cannot  be  ergodic,  which  contradicts  our  assumption.  This 
result  completes  the  proof  of  necessity.  □ 

We  now  return  to  the  discussion  of  the  stability  of  the  embedded  chain  Q*(oo), 
which  we  have  shown  to  be  a  Markov  chain  of  M/G/l-type.  In  order  to  facilitate  the 
derivation  of  simple  analytic  criteria  for  Q*( oo)  to  be  positive  recurrent,  we  define 
the  (limiting)  matrices 

OO 

A  =  lim  and  (3  =  lim  (3^  =  lim  uA //\ 
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and  the  vector 


7T  =  lim  7T  *  , 

i — >oo 

assuming  that  the  limits  exist.  It  is  then  true  by  the  fundamental  properties  of  limits 
that 

p  =  lim  7 j-W/^Wg  —  7r/3e, 

i— »oo 

or,  in  other  words,  we  may  first  compute  individual  limits,  then  multiply  the  terms 
together  in  order  to  obtain  p.  Thus,  it  remains  to  determine  the  methods  by  which 
we  compute  the  matrices  (3^  and  the  invariant  probability  vectors 

The  determination  of  (3^  is  contingent  on  making  the  observation  that,  since 
the  matrix  ^-transform,  A^*  (z)  =  'Y^=QA$zv,  is  analytic  on  the  unit  disk  \z\  <  1, 
we  may  interchange  the  order  of  the  derivative  and  the  summation  to  obtain 

7  OO  7  OO 

j-A('rU)  =  =  T  vAfz"-\  (5.52) 

u=0  U  v= 0 

and  thus  we  may  obtain  the  matrix  /?W  by  taking  the  derivative  of  the  matrix- 
transform  A^*(z)  and  evaluating  at  z  —  1,  as  follows: 

(5.53) 

Z=1 

The  vector  7T  =  lim^oo  7r^,  where  the  are  the  invariant  vectors  of  the 
matrices  A^l\  is  obtained  by  noting  the  following,  which  is  that 

7r  =  lim  7r^  =  lim  tt^A^  =  ~k A  (5.54) 

xx)  i — >oo 

and 

1  =  lim  7 r^e  =  7re,  (5.55) 


/3(i)  =  ^-A®\z) 
dz 
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and  thus  7r  is  the  invariant  probability  vector  of  the  stochastic  matrix  A.  The 
computation  of  A  itself  may  be  accomplished  via  the  following: 

OO  OO 

a®  =  J2  =  J2  A^zV  =  A(i) *  (x) 

!/=0  V=0  z=l 

(see  (5.46)  for  the  definition  of  A*(z)).  We  thus  obtain 


A  =  lim  7lw*(l). 


(5.56) 


5.3.1  Computation  of  the  Traffic  Intensity 

With  the  theoretical  background  afforded  by  Theorem  5.3  we  may  now  proceed 
to  determine  the  values  of  the  parameter  vectors  A,  £,  a,  6 ,  and  the  scalar-valued 
average  service  time  1/ p  that  guarantee  the  validity  of  the  following  statements: 

OO 

lim  7 =  lim  7 uAAe  <  1,  (5.57) 

i—>  oo  i—>  oo  /.--j 

v= 0 

oo 

vBv  <  oo.  (5.58) 

u=0 

Through  unconditioning,  and  by  the  definition  of  T^(A;,0),  we  obtain 

A®  =  0-59) 

k= 1 

Define 

w  =  7®(,)*(m,2)  „0. 

az  z= 1 

which  is  equivalent  to  the  expected  number  of  primary  arrivals  to  the  queue  in  a 
(regenerative)  cycle  given  that  there  are  i  customers  in  the  system  (orbit).  If  one 
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factors  in  the  observation  that 


d 


p(i)  =  £>4?  =  Tza{‘)’{3'z) 


u=0 


d 


s=0  ’ 

Z=1 


^VB-  =  izB‘(S’Z 


v= 0 


s=0  ’ 

Z=1 


it  becomes  clear  that  we  may  rewrite  (5.57)  and  (5.58)  as 


(5.60) 


(5.61) 


lim  7r(i)/3(?:)e  =  lim  7r(i)  <|  Vl#  P® 


/  re’ 

k=  1 


(5.62) 


and 


E*B-  =  Evfff  < 


__  oo.  (5.63) 

u=0  k= 1 

Starting  from  (5.56),  we  obtain  the  invariant  probability  vector  of  the  stochastic 
matrix  A  as  follows: 


A  =  Aw*(l)  =  lim  yv^ 

i — wyi  •  ^ 


i)l 


u=0 


s=0 
z= 1 


From  (5.59),  we  thus  obtain 


A  = 


iimyyy'*<‘)(MW 

r — 


(0 


v= 0  k= 1 


s=0 
*=1  fc=l 


£  *“>*(*,  0,1)P, 


« 


(5.64) 


The  interchange  of  summations  in  (5.64)  is  contingent  upon  the  analyticity  of 
T^(£;,  s)  in  a  neighborhood  of  s  =  0. 


5.3.2  Limiting  Values  of  P^ 

We  begin  by  considering  the  terms  P/c  =  lim*-^  P^  for  cycle  types  k  = 
1,  2, . . . ,  5.  To  this  end,  we  present  the  following  result: 
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Proposition  5.1. 


lim  (Tv  ©  Ti6)  =  Ihn  (Ti0  ©  Tv)  =  0, 


where  Tv  =  Q  —  A (77)  for  some  exponential  rate  parameter  77. 


Proof.  We  recall  from  Lemma  A.l  that 


(T.n  ©  T^)  =  /  exp  [(—Tv  ©  Tie)x]  dx 


exp (—Tvx)  ©  exp  (Ti0x)  dx 


exp (—Tvx)  ©  exp  [(Q  —  iA(6))x]  dx. 


since  Q  and  A(0)  commute,  we  may  further  simplify  the  above  to  obtain 


(Tn  ©  Te)  =  /  exp(— Tvx)  ©  exp(Qa;)  exp  [— iA(9)  x]  dx.  (5.65) 

Jo 


Since  we  have,  for  diagonal  matrices  A(rj),  the  identity 


exp(A(r7))  =  A  (exp (77)) 

where  exp(77)  =  [er>i\je{i,...,m}i  (5.65)  becomes 

exp(— Tvx)  ©  exp(Qa:)  A  ( e~l6x )  dx  —>  0 

as  i  — >  00,  and  thus  lim^oo  ( Tn  ©  T^)-1  =  0.  □ 

Proposition  5.1  allows  us  to  prove  the  following  for  a  generic  vector  a: 
Proposition  5.2.  Given  a  (constant)  generic  MMPP  rate  vector  a  and 

P0(tO,a)  =  P{Xie  <  Xa)  , 
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its  limit  as  i 


oo  is  given  by 


lim  P0(iO,  a)  =  lim  [1  +  (4>ie  ®  <f)a)(-Tie  ©  Ta)  l(e  ®  (Tae)\  =  1.  (5.66) 

2— XX)  i — >oo 

Proof.  The  result  immediately  follows  from  the  application  of  Proposition  5.1.  □ 

The  next,  and  last,  result  resolves  an  issue  concerning  the  limiting  value  of  a 
certain  type  of  expression  containing  MMPP-rclated  elements: 

Proposition  5.3.  Given  a  (constant)  generic  MMPP  rate  vector  a  and  a  generic 
vector  7,  we  obtain  the  limiting  value 


lim(0j0®7 ){-Tie@Ta)  l(Ti0e®e)  (5.67) 


=  lim  (<p  ®  <f>i0)(-Ta  ©  Tie )  X(e  ©  Ti0e) 

n  — vrvo  ' 


=  — 'ye. 


Proof.  As  in  Proposition  5.1,  we  rewrite  the  argument  of  the  limit  in  (5.67)  in  integral 
form  and  then  apply  the  technique  of  partial  integration: 


{4>i8®l){Tie@Ta)  1(Tiee®e) 

poo 

=  /  (<f>i0exp(-Tigt)Tige)('Yexp(Tat)e)  dt 
Jo 

oo 

=  (0j:eexp(-Ti0f)e)(7exp(Tat)e) 

t= o 


^  {<t>ie®  l)  j  exp ((Tie®TP)t)dt(e®Tae) 


=  -(0*0e)( 7e)  -0  =  -7e. 


□ 
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Through  the  application  of  Proposition  5.1,  one  may  conclude  that 


Pk  =  lim  P®  =  0,  k  —  1,2,4,  (5.68) 

i— >  oo 

since  each  are  multiplied  by  a  probability  of  the  form  P0(a,  id ),  where  a  is  a  generic 
MMPP  rate  vector.  For  Cases  3  and  5,  Proposition  5.3  may  be  used  to  show  that 


P3  =  lim  P3W 

i— xx) 

=  0{(/-A(-T{))e 

P5  =  lim  P5W 

i— xx 

=  1  -  4>(  (I  -  e. 

(5.69) 

This  result  confirms  the  intuitive  notion  that  Case  3,  which  is  the  interruption  of 
the  service  of  a  retrial  customer,  and  Case  5,  the  completion  of  a  retrial  service, 
are  the  key  events  that  influence  system  stability  as  the  orbit  size  becomes  large. 
Further  revealing  is  the  observation  that  the  scenario  of  Case  5  is  the  only  one 
that  permits  the  orbit  size  to  decrement;  thus,  at  a  minimum,  the  probabilities  of 
transition  between  orbit  sizes  resulting  from  the  situation  of  Case  5  must  be  taken 
into  consideration  for  all  i  >  1. 

5.3.3  The  Ergodicity  of  { ( Rn ,  Zn)  :  n  >  0} 

Now  that  the  requisite  facts  and  concepts  have  been  presented,  we  may  com¬ 
pute  an  explicit  traffic  intensity  formula  for  the  embedded  chain  {(Rn,  Zn)  :  n  >  0}. 
We  shall  first  state  the  result,  and  then  provide  the  actual  steps  of  the  computation 
as  a  proof. 

Theorem  5.4.  The  embedded  Markov  chain  {(Rn,  Zn)  :  n  >  0}  is  ergodic  if  and 
only  if  both  of  the  following  conditions  hold: 

1.  The  overall  traffic  intensity,  p,  fulfills  the  condition 

p  =  p(3e  <  1, 
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where  p  is  the  steady-state  probability  vector  of  the  random  environment,  (3  is 
the  matrix  given  by 


P  = 


U-Q)  +  t3(a(a))  \p3  +  {  fM-Tx(z)) 


Z=  1 


P5i  (5.70) 


and 


r3(X)  =  (10  <j>z)  [(Q  ©  T^)-\X  ©  TP)(Q  ©  T^y1]  (I  0  T{ a 


(I0cf>a0  0€)  [(<?©/©  y)-\X  0  I  ©  Tz)(Q  0  I  ©  Te)-r 


x  [(/  ©  (~Ta)e  0e)  +  (l0e0  T^ej 


-[(Q®Ta)-1(X@Ta)(Q@Ta)  1  0  I)](1 0Tae0e)\.  (5.71) 


2.  The  condition, 


(/  ©  0J(Q  ©  Ta)~H  (/  ©  Tae)  +  (A(A)  ©  Ta)(Q  ©  Ta) 


-i 


x  (I  0  Tae)  P[0)  +  {  2T 2(-Q)  +  T2(A(A))  |>P2l 


>(0) 


\(-Q)  + 


z=l 


P5(0)  <  cx),  (5.72) 


where 


T2(X)  =  |(/©^©0A)  [(Q  ©  T^iX  ©  Tf)(Q  ©  P^)"1  ©  I] 


x  [/  ©  (Tc  ©  /)]  >/Pq(A,  0  -  (/  ©  0Q  ©  ^  ©  0J 
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x  [(g®/©^)-1^® /©^(g®/©:^)-1® /] 


x  {(/  <g>  (~Ta)e  ©(/©/))  +  (/©  e  <g>  (Te  ©  /))  +  (/  ©  Tae  ©  (/  ©  /))} 

-  [(g  ©  Ta)~\X  ©  Ta){Q  ©  Ta)~l  ©  (/  ©  /)] 

x  (/©Tae©  (/®/))j(-r?©TA)“1(e®TAe).  (5.73) 

Proof.  This  result  is  a  straightforward  application  of  Theorem  5.3,  which  provides 
the  basic  criteria  for  the  ergodicity  of  any  M/G/l-type  DTMC.  The  first  stipulation 
of  the  theorem  is  that 

7 r/3e  <  1, 

where  7r  is  defined  as  the  invariant  probability  vector  of 

OO 

A  =  Inn  V  A®. 

i— xx)  '  J 
u=0 

It  is  clear  from  this  definition  that  A  gives  the  steady-state  transition  probabilities 
of  {( Rn ,  Zn)  :  n  >  0}  aggregated  by  environment  state,  which  is  the  same  regardless 
of  the  number  i  in  orbit.  Thus,  7 v  =  p,  which  is  interpreted  as  the  steady-state 
probability  vector  of  the  random  environment  embedded  at  regenerative  epochs. 

We  next  use  (5.62)  as  a  basis  for  the  computation  of  p  in  condition  (1),  while 
applying  the  results  for  the  limiting  probabilities  of  Cases  1  through  5  given  in  (5.68) 
and  (5.69)  in  order  to  state 

5 

(3  =  lim  (3{i)  =  lim  V  mP  P  f 

2— XX)  Z — >OG  '  J 

k=  1 

=  lim  ( Mf  P3(i)  +  M5(i)  P f) 

2— XX)  \  / 
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=  lim 


^  ($«*>*  (3,  s,  z)  pf  +  <!>«*  (5,  s,  *)  If) 


s=0 

2=1 


(5.74) 


Tedious  algebra  then  yields  the  expressions  given  in  (5.70)  and  (5.71). 

Subsequently,  for  condition  (2),  we  refer  to  the  second  criterion  of  Theorem 
5.3,  which  is  that 

OO 

^  isBu  <  oo. 

u=0 

As  in  (5.61),  we  compute  this  quantity  in  the  following  manner: 


Y.vD‘ 

17=0 


(o) 


k= 1 


s=0 

2=1 


,s,z)P^  +  ^°>(5,s,z)P^) 


s= 0  * 

Z=1 


This  likewise  yields  the  quantities  (5.72)  and  (5.73),  which  estabilishes  (2)  as  the 
remaining  criterion  for  the  ergodicity  of  {(Rn,  Zn)}.  We  have  thus  shown  that  con¬ 
ditions  (1)  and  (2)  are  the  necessary  and  sufficient  requirements  for  the  ergodicity 
of  {(Rn,  Zn ) }  as  specified  by  Theorem  5.3.  □ 

As  with  the  traffic  intensity  formula  (4.18),  all  terms  containing  6  disappear 
in  the  limit.  This  is  in  accordance  with  results  for  all  other  retrial  queues,  which  is 
that  the  retrial  rate  has  no  bearing  upon  the  long-term  stability  of  the  queue.  We 
may  use  Theorem  5.4  to  interpret  this  result,  which  is,  plainly  stated,  that  system 
stability  is  measured  against  the  worst  possible  scenario,  namely  that  in  which  the 
orbit  size  grows  very  large.  However,  as  the  orbit  size  grows  large,  the  retrial  rates 
also  grow  without  bound,  until  the  system  becomes  an  instantaneous-feedback  retrial 
queue  (i.e.,  from  the  point  of  view  of  the  server,  retrials  occur  continuously).  It  is 
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thus  apparent  that  a  retrial  will  obtain  service  with  probability  one,  regardless  of 
the  value  of  the  individual  retrial  rate. 


5.3.4  Application  to  the  Exponential  Model 

Theorem  5.4  applies  to  the  embedded  Markov  chain  {(Rn,  Zn)  :  n  >  0}  of 
{(R(t),  Z(t),  X(t))  :  t  >  0}  at  the  regenerative  epochs  Tn  so  long  as  the  service 
distribution  meets  the  requirements  stated  in  Section  5.1.  We  may  thus  com¬ 
pare  the  stability  criteria  for  the  general  model  to  that  with  exponential  service. 
More  specifically,  we  consider  the  retrial  qneneing  model  of  [62]  that  excludes  the 
possibilities  of  customer  balking  and  Markov  modulation,  with  the  additional  as¬ 
sumption  of  exponentially-distributed  service  times.  This  model  is  equivalent  to 
{(R(t),  Z(t),X(t))  :  t  >  0}  if  one  fixes  all  of  the  exponential  rates  of  arrival,  service, 
failure,  repair,  and  retrial.  Any  random  environment  that  is  an  irreducible  Markov 
chain  may  be  thus  chosen  for  the  comparison. 

The  parameters  of  the  model  itself  are  not  numerically  specified.  They  are 
simply  left  in  terms  of  their  symbolic  representations  A,  /i,  f,  a,  and  6 ,  as  defined 
in  Sections  4.1  and  5.1.  The  density  function  h(x )  of  the  service  distribution  is  thus 
chosen  to  be 

h(x )  =  pe~^x, 

which  is  the  exponential  p.d.f.  with  rate  p.  We  define  the  infinitesimal  generator  Q 
of  the  random  environment  as  the  2x2  matrix 


Q  = 


-1 

2 


whose  steady-state  probability  vector  is  thus  given  by  p  =  [2/3, 1/3].  Lastly,  all  of 
the  initial  probabilities  <f>ri.  where  77  is  the  rate  vector  of  a  MMPP,  are  fixed  at  p. 
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The  next  task  is  to  obtain  the  limiting  probabilities  for  Cases  3  and  5.  Direct 


computation  yields 


P,  = 


v  +  £ 


and  Pk  = 


A1  +  £ 


The  corresponding  2x2  transform  matrices 


lim  0,  z),  k  =  3,  5, 


are  found  to  be 


Tw*(3,0,;?)  = 


za 


(A(l  —  z)  +  a)(A(l  —  z)  +  a  +  3) 


A(1  —  z)  +  a  +  2  1 

2  A(1  —  -2)  T  ol  T  1 


and 


T^*(5,  0,  z)  = 


/i  +  3 


l-i  +  2  1 

2  /i  +  1 


In  order  to  compute  the  matrix  /3,  it  is  sufficient  to  obtain 


lim  AfW*(3,0,l)  =  —  lim  T(i)*(3,  0,  z) 

i— >00  a£  i— >00 


2=1 


since  0,  z)  is  constant.  Consequently, 


/ 3  =  lim  Mw*(3,0,1)P3W  =  -j-  lim  tfW*(3,  0,  z)P^ 

i—> 00  dz  i—> 00 

= _ _ 

(/i  +  £)(A(1  —  z)  +  /i)2A(l  —  z)  +  \i  +  3)2 


2=1 


h(z)  (2A(1  -  z)  +  2/x  +  3)2 

—  (2A(1  —  z)  +  2/i  +  3)2  k-2^z) 
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where 


ki(z)  —  -(A(l  —  z)  +  /x  +  3)2  +  -(A(l  —  z)  +  /x)2, 
^2(2)  =  g(A(l  —  z)  +  /x  +  3)2  +  -(A(l  —  2)  +  /x)2. 


The  traffic  intensity  expression  may  finally  be  computed  as 


p  =  p(3e 


+  A(q  + 

a(/x  +  0 


(5.75) 


Notice  that  (5.75)  is  exactly  the  conditional  traffic  intensity  formula  that  can  be 
derived  from  [62:  Thm  1],  and  which  appears  in  (4.34). 

This  concludes  the  discussion  of  the  steady-state  analysis  of  the  unreliable 
M/G/l  retrial  queue  in  a  random  environment.  We  were  able,  using  the  matrix- 
analytic  theory  of  Markov  chains  of  M/G/  1-type,  to  obtain  the  transition  prob¬ 
ability  matrix  for  embedded  Markov  chain  of  a  complex  retrial  queueing  system. 
We  subsequently  proved  a  condition  for  the  ergodicity  of  general,  level-dependent, 
discrete-time  M/G/ 1-type  Markov  chains.  To  the  best  of  our  knowledge  these  results 
are  novel.  We  then  utilized  this  condition  in  order  to  derive  a  stability  expression 
for  the  unreliable  Markov-modulated  M/G/l  retrial  queueing  system. 
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6.  Queueing  Model  Optimization 

The  results  of  Chapter  4  may  be  used  to  improve  the  performance  of  retrial  queueing 
systems  through  the  optimization  of  design  or  operating  parameters.  However,  the 
matrix-analytic  approach  taken  in  Chapter  4  leads  to  a  numerical  approximation  of 
the  steady-state  orbit  size  distribution;  therefore,  a  closed-form  objective  function 
of  the  operating  parameters  is  not  available.  We  therefore  resort  to  mesh-adaptive 
search  techniques  for  problems  that  have  no  derivative  information.  For  such  tech¬ 
niques,  it  is  not  necessary  to  specify  a  closed-form  objective  function,  so  long  as 
the  objective  function  may  be  numerically  evaluated  at  points  inside  the  feasible  re¬ 
gion.  For  detailed  information  on  the  search  algorithms  applied  in  this  chapter,  the 
reader  is  referred  to  the  excellent  summaries  contained  in  [2,  15,  16].  This  chapter 
is  illustrative  in  nature,  highlighting  the  potential  usefulness  of  the  main  results  (for 
the  exponential  model)  in  improving  the  performance  of  retrial  queues  with  complex 
dynamics. 

6.1  Problem  Formulation 

A  principal  concern  of  most  queueing  systems  is  the  average  amount  of  time 
customers  spend  in  the  retrial  orbit.  For  instance,  retrial  queues  can  be  used  to 
model  customer  contact  centers  wherein  customers  who  dial  into  the  center  may  not 
receive  service  immediately  and  try  back  after  a  random  time.  A  critical  performance 
measure  is  the  expected  time  such  customers  spend  outside  the  system  attempting  to 
gain  service.  An  intrinsic  cost  (that  may  be  difficult  to  ascertain)  may  be  associated 
to  each  unit  of  time  that  the  customer  spends  in  the  orbit.  Thus,  we  consider 
the  minimization  of  E[Wft] .  The  decision  variables  are  the  exponential  arrival  and 
service  rates  defined  by  the  vectors  A  =  [Aj]  and  =  [/q],  respectively.  We  denote 
by  p(A,  n),  the  traffic  intensity  defined  in  (4.18),  now  expressed  as  a  function  of  the 
decision  variables  A  and  /i. 
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As  before,  the  integer  m  denotes  the  number  of  distinct  random  environment 
states,  and  we  define  the  nonnegative  row  vectors  aq,  x2,  yl7  and  y2  such  that  for 
3  =  1,2,. .  -,m, 


Xi  = 

[aqi,  X12,  ■ 

■  ■  ,  xlj]i 

x2  = 

[^21,  X22,  ■ 

■  ■  ,  %2j\ , 

2/i  = 

[2/11,2/12,  •  • 

•  5  2/ij] , 

2/2  = 

[2/21,  x22, .  , 

•  • ,  2/2 j] 

with  X\  <  x2  and  yx  <  y2.  We  may  then  state  the  mathematical  programming 
formulation  as  follows: 


min  E[Wr] 

s.t.  p(A,  n)  <  1 

(6.1a) 

Xj  e  [x^x^],  j  =  1,.. 

. . ,  m 

(6.1b) 

hj  ^  [  2/11,2/21],  j  1,  •  • 

. ,  m. 

(6.1c) 

It  should  be  recognized  that  the  objective  function  E[Wr]  may  only  be  evaluated 
numerically,  while  constraints  (6.1b)  and  (6.1c)  are  box  constraints  that  are  provided 
by  the  user.  The  only  quantity  that  may  be  computed  explicitly  from  the  model  is 
the  left-hand  side  of  constraint  (6.1a),  which  is  the  traffic  intensity  (4.18). 

6. 2  Solution  Procedure 

The  optimization  software  NOMADm  [1]  was  employed  to  solve  problem  for¬ 
mulation  (6.1).  The  advantage  of  this  approach  rests  in  the  intrinsic  flexibility  of  this 
software,  which  can  locate  high-quality  solutions  given  only  a  numerically  defined 
objective  function,  a  set  of  constraints  and  an  initial  feasible  solution.  NOMADm  em¬ 
ploys  a  form  of  generalized  pattern  search  (GPS)  algorithm  known  as  mesh-adaptive 


132 


direct  search  (MADS),  that  incrementally  explores,  or  polls ,  the  surrounding  feasible 
region  at  each  iteration  of  the  procedure.  The  iterations  of  the  algorithm  are  con¬ 
ducted  on  a  mesh ,  which  is  a  means  by  which  the  increments  of  steps  through  the 
feasible  region  is  controlled.  The  mesh  may  be  coarsened  or  refined  as  the  situation 
dictates,  hence  the  term  mesh- adaptive.  The  first  step  is  entitled  search ;  any  point 
on  the  mesh  may  be  evaluated  during  a  search  routine,  which,  as  stated  in  [2],  can 
be  defined  in  any  manner  whatsoever.  The  poll  step,  however,  is  essential  to  the 
workings  of  the  algorithm.  It  involves  the  search  of  points  on  the  mesh  that  neigh¬ 
bor  the  current  iterate.  Most  of  the  theoretical  focus  regarding  generalized  pattern 
search  concerns  the  effect  of  various  polling  strategies  upon  rate  of  convergence  of 
the  algorithm  and  the  quality  of  the  solutions. 

Because  MADS  is  stochastic  in  the  way  that  it  polls  surrounding  points,  it 
is  instructive  to  conduct  multiple  optimization  runs  for  the  same  problem  instance. 
The  user  may  alleviate  the  computational  cost  of  such  additional  runs  by  storing 
previously  visited  points  in  a  cache  hie,  which  thus  allows  faster  run  times,  though 
at  the  expense  of  hard-disk  storage.  Selecting  different  initial  feasible  points  for  the 
algorithm  may  also  increase  confidence  that  a  global  rather  than  local  minimum  has 
been  obtained.  A  suitably  chosen  search  routine,  in  conjunction  with  polling,  may 
likewise  be  used  to  mitigate  this  risk. 

It  should  be  noted  that,  in  the  numerical  implementation  of  (6.1),  constraint 
(6.1a)  is  not  included  as  a  formal  constraint.  Rather,  any  points  (X,  pi)  that  render 
the  system  unstable  are  simply  not  investigated  any  further  during  a  polling  (or 
search)  step.  The  algorithm  removes  such  points  from  its  list  of  solution  candidates 
and  then  proceeds  to  the  next  trial  point  on  the  mesh.  This  is  done  as  a  precaution 
to  ensure  that  the  routine  does  not  attempt  to  evaluate  the  objective  function  at 
points  that  lead  to  instability  of  the  queueing  system.  In  such  cases,  the  steady-state 
probabilities  do  not  exist,  and  hence,  we  avoid  further  problems  that  may  arise  from 
this  circumstance. 
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6.3  Numerical  Illustrations 


The  problem  instances  presented  in  this  subsection  are  exponential  with  3- 
and  5-state  random  environments,  respectively.  Initial  feasible  vectors  Ao  and  n0 
were  specified  for  each  of  the  3-  and  5-state  cases.  The  remaining  vectors,  £,  ol  and 
6,  were  assigned  values  that  did  not  change  during  optimization  runs.  Whenever 
a  run  terminated,  the  values  of  A*  and  /i*  that  produced  the  optimal  cost  E[Wr]* 
were  recorded.  Additional  runs,  using  distinct  initial  feasible  points,  were  used  to 
help  ensure  that  the  algorithm  produced  consistent  solutions.  As  noted  above,  the 
objective  is  to  minimize  the  steady-state  mean  time  spent  in  orbit,  E[Wr]. 


6.3.1  Three-State  Environment 

We  now  present  results  for  the  NOMADm  optimization  of  the  queueing  model 
subject  to  a  three-state  random  environment  with  infinitesimal  generator  matrix 


Q  = 


-2.0 

1.0 

1.0 


1.0 

-2.0 

1.0 


1.0 

1.0 

-2.0 


(6.2) 


The  values  of  the  parameters  «,  and  6  were  fixed  at  the  values  shown  in  Table 
6.1.  In  addition,  we  specify  the  box  constraints  on  the  decisions  variables  A  and  /i 
to  be 


<  Aj  <  6,  j  =  1,2,3 

(6.3) 

<H<  8,  3  =  1,2,3. 

(6.4) 

Results  for  a  single  optimization  run  appear  in  Table  6.2.  These  intuitive 
results  indicate  that  the  optimal  parameters  A*  and  /i*  lie  mostly  on  the  boundaries 
of  the  region  defined  by  (6.3)  and  (6.4).  This  is  a  consequence  of  the  fact  that  no 
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Table  6.1  Problem  data  for  3-state  example. 


Vector 

State  1 

State  2 

State  3 

^0 

2.0 

3.5 

4.0 

Mo 

4.0 

3.5 

7.0 

3.0 

2.0 

0.1 

a 

4.0 

7.0 

2.0 

e 

1.5 

3.0 

5.0 

penalties  were  assessed  in  the  problem  formulation  for  increasing  service  rates  or  for 
decreasing  customer  traffic.  An  exception  to  this  was  the  value  of  A3,  which  lies 
in  the  interior  of  the  region  defined  by  the  box  constraints  (6.3)  and  (6.4).  That 
A3  >  1  was  not  binding  is  due  to  the  intersection  of  the  region  enclosed  by  the  box 
constraints  with  that  defined  by  the  constraint  p( A,  n)  <  1. 

We  were  likewise  interested  in  the  convergence  rate  of  MADS.  A  plot  of  the 
iteration  history  (see  Figure  6.1)  revealed  that  most  of  the  optimality  gap  is  elim¬ 
inated  by  MADS  in  the  first  50  of  389  iterations.  This  information  is  needed  for 
estimating  truncation  points  for  individual  runs,  particularly  as  the  number  of  envi¬ 
ronmental  states  increase.  We  shall  obtain  a  glimpse  of  the  dimensionality  problem 
in  the  five-state  example  that  we  discuss  next,  for  which  computation  time  becomes 
a  significant  issue. 


Table  6.2  Optimal  solution  for  3-state  example. 


Vector 

State  1 

State  2 

State  3 

E[Wr}* 

A* 

1.00 

1.00 

1.71 

6.9014 

M* 

8.0 

8.0 

8.0 
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Figure  6.1  Graphical  depiction  of  the  sequence  of  MADS  iterations  for  the  three- 
state  example. 


6.3.2  Five- State  Environment 


We  now  present  an  example  using  a  five-state  random  environment  with  in¬ 


finitesimal  generator  matrix 


Q 


-11000 
0-1100 
0  0-11  0 
0  0  0  -1  1 

1  0  0  0  -1 
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The  box  constraints  for  the  environment-dependent  arrival  and  service  rates  are 
given  by 


1  <  A j<  5, 

3  —  1)2,. 

..,5 

(6.5) 

VI 

a. 

VI 

o 

3  =  1,2,.. 

.,5, 

(6.6) 

and  the  remaining  problem  data  is  summarized  in  Table  6.3. 

Table  6.3  Problem  data  for  5-state  example. 


Vector 

Environment  State 

1 

2 

3 

4 

5 

Ao 

1.0 

1.0 

1.0 

2.0 

5.0 

Mo 

2.0 

2.5 

1.0 

3.0 

4.0 

0.5 

1.1 

1.5 

4.0 

1.0 

at 

2.0 

0.5 

8.5 

4.5 

6.0 

e 

1.0 

4.0 

2.0 

8.0 

5.0 

Table  6.4  summarizes  the  optimal  solution  vectors  and  objective  function  value 
for  this  illustration.  The  rate  of  convergence  seen  in  Figure  6.2  is  similar  to  that  of 
the  three-state  model.  Again,  the  optimal  vectors  A*  and  n*  lie  almost  exclusively 
on  the  boundaries  of  the  region  defined  by  the  box  constraints  in  (6.5)  and  (6.6), 
with  the  exception  of  A5,  which  is  set  at  its  upper  bound,  and  thus  Ag  >  1  turns  out 
to  be  a  nonbinding  constraint.  Again,  this  is  due  to  the  fact  that  penalties  are  not 
assessed  for  choosing  arrival  and  service  rates  on  the  boundary  of  the  feasible  region. 

The  processing  time  required  to  complete  the  optimization  run  was  much  more 
substantial  here  than  for  the  model  with  a  three-state  random  environment.  This 
was  a  consequence  of  the  exponential  increase  in  the  computational  effort  needed  to 
compute  the  steady-state  probabilities  at  each  objective  function  evaluation. 
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Table  6.4  Optimal  solution:  5-state  example. 


Vector 

Environment  State 

E[Wr}* 

1 

2 

3 

4 

5 

A* 

* 

1.0000 

3.9996 

1.0000 

3.9229 

1.0000 

3.9999 

1.0000 

3.9999 

4.9998 

4.0000 

13.279 

The  availability  of  the  steady-state  distribution  in  numerical  form  allows  us  to 
obtain  approximately  optimal  operating  parameters  for  the  M/M/1  retrial  queue¬ 
ing  system.  We  have  demonstrated  that  the  computational  method  outlined  in  this 
chapter  can  be  applied  to  a  variety  of  complex  queueing  systems,  so  long  as  the  limit¬ 
ing  probabilities  can  be  obtained.  This  includes  the  large  class  of  queueing  models  to 
which  the  matrix- analytic  techniques  may  be  applied.  While  many  other  aspects  of 
the  optimization  of  unreliable  retrial  queueing  systems  should  be  investigated  (e.g., 
admission  control,  prioritized  retrial  orbit,  etc.),  our  main  purpose  here  was  to  il¬ 
lustrate  the  feasibility  of  using  the  main  results  to  determine  operating  parameters 
(namely  the  arrival  and  service  rates)  that  minimize  the  steady-state  mean  time  in 
orbit.  In  the  final  chapter,  we  review  the  main  contributions  and  conclusions  of  this 
research  and  provide  some  important  directions  for  future  work. 
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7.  Contributions  and  Future  Research 

In  this  dissertation,  much  progress  has  been  made  toward  the  understanding  of 
modulated  retrial  queueing  models  subject  to  breakdowns.  Although  a  number  of 
researchers  have  studied  the  steady-state  performance  of  retrial  queues,  this  disser¬ 
tation  is  the  first  to  consider  Markov  modulation  and  server  breakdowns  in  a  single 
model.  Moreover,  matrix- analytic  methods  were  employed  in  the  mathematical  char¬ 
acterization  of  the  stability  of  the  M/M/1  and  M/G/l  versions,  as  well  as  to  their 
subsequent  numerical  solution. 

Chapter  4  first  considered  the  foundational  case  of  a  single-server  system  with 
Poisson  arrivals  and  Markov  modulation  subject  to  server  failures.  For  this  queue¬ 
ing  model,  a  formal  stability  analysis  was  provided  that  has  further  implications  for 
more  general  level-dependent  quasi-birth-and-death  (LDQBD)  processes.  By  em¬ 
ploying  classical  techniques  and  the  matrix-analytic  framework,  a  computable  traffic 
intensity  was  derived  as  an  explicit  function  of  the  Markov-modulated  arrival,  ser¬ 
vice,  retrial,  failure  and  repair  rates.  We  further  illustrated  (in  Chapter  6)  how  the 
results  of  this  chapter  can  be  used  to  pragmatically  select  arrival  and  service  rates 
that  minimize  the  steady-state  mean  time  a  customer  spends  in  orbit,  an  impor¬ 
tant  measure  in  a  number  of  applications  including  communications  and  computer 
networks  as  well  as  customer  contact  centers. 

We  then  set  out  in  Chapter  5  to  devise  similar  conditions  for  the  stability  of 
the  same  retrial  queueing  system  when  each  customer  brings  a  generally  distributed 
service  requirement.  In  this  model,  the  service  distribution  is  not  modulated  by  the 
random  environment,  but  all  other  processes  operate  exactly  as  before.  In  this  case, 
it  was  shown  that  the  Markov  chain  embedded  at  epochs  in  which  the  server  enters 
the  up  and  idle  state  is  a  Markov  chain  of  M/G/l  type.  Consequently,  we  were  able 
to  prove  the  conditions  needed  for  stability  of  the  queueing  system.  Furthermore, 
through  some  tedious  mathematics,  we  were  able  to  explicitly  define  the  transition 
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probability  matrix  of  the  embedded  Markov  chain  and  define  all  of  its  elements  in 
the  matrix- analytic  format.  To  the  author’s  knowledge,  such  results  have  appeared 
neither  in  the  stochastic  modeling  nor  the  retrial  queueing  literature. 

While  this  dissertation  contributes  several  basic  results  for  unreliable  retrial 
queueing  systems  that  operate  in  a  random  environment,  there  are  several  potentially 
fruitful  avenues  for  extension  of  this  research.  One  obvious  extension  is  to  consider 
the  exponential  model  with  multiple  servers.  Such  a  queueing  system  can  be  used 
to  model  the  operation  of  customer  contact  centers  which  have  recently  received  a 
great  deal  of  attention  in  the  stochastic  modeling  community.  In  such  case,  one 
may  consider  customers  who  also  enter  the  retrial  orbit  by  abandoning  the  system 
if  their  wait  time  exceeds  some  predetermined  threshold.  It  may  also  be  instructive 
to  consider  non-exponential  inter-retrial  times  since  customers  may  retry  the  system 
at  arbitrary  random  intervals. 

Another  important  extension  of  this  work  is  the  consideration  of  networks 
of  unreliable  retrial  queueing  systems  in  a  random  environment.  These  may,  for 
example,  be  used  to  model  Internet  traffic  wherein  the  various  environment  states 
may  correspond  to  the  traffic  in  the  network  which  impacts  the  performance  of  each 
individual  queueing  station.  Analyzing  a  stochastic  network  of  this  kind  promises  to 
be  very  challenging,  though  worthwhile. 

The  optimization  portion  of  this  research  served  the  purpose  of  illustrating  how 
the  results  might  be  used  to  improve  the  performance  of  a  retrial  queueing  system 
subject  to  time-varying  conditions.  It  will  be  instructive  in  the  future  to  consider 
budget  constrained  systems  which  add  significant  complexity  to  the  optimization 
model  of  Chapter  6.  Furthermore,  dealing  with  a  “black  box”  objective  function 
presents  numerical  challenges  that  need  to  be  addressed  since  it  is  difficult  to  discern 
when  a  global  optimal  solution  to  the  problem  exists.  The  computational  complexity 
involved  in  evaluating  candidate  solutions  is  not  a  minor  issue.  New  algorithms  for 
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efficiently  computing  the  approximate  steady-state  orbit  size  distribution  will  be 
needed  to  help  address  this  issue. 
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Appendix  A. 

A.l  Matrix  Binary  Operators 

We  review  here  some  basic,  but  important,  operations  on  the  space  of  matrices 
K mxn  that  we  shall  require  in  order  be  able  to  apply  the  matrix-analytic  methods 
to  the  study  of  quasi-birth-and-death  processes  (QBDs),  the  M/G/  1-type  Markov 
chains,  and  Markov  modulation.  We  shall  first  discuss  Kronecker  products  and  sums, 
which  are  crucial  to  the  manipulation  of  matrix  expressions  under  integration.  We 
also  discuss  another  binary  operator  termed  the  matrix  convolution  product,  which 
is  the  matrix  version  of  the  convolution  of  scalar  distribution  functions. 


A.  1.1  The  Kronecker  Product  and  Sum  of  Matrices 

A  useful  tool  in  the  representation  of  block-matrix  expressions  is  the  binary 
operator  (8)  :  Mmxn  x  Mpx<?  — >  R(mP)x(n9) ,  which  is  known  as  the  Kronecker  product, 
where  Mmxn  denotes  the  set  of  m  x  n  matrices  over  the  field  of  real  numbers.  The 
Kronecker  product  is  also  known  as  the  tensor  product  when  used  in  a  more  general 
algebraic  or  topological  context.  The  operation  is  defined  as  follows: 


anB 

Q 1 2  B 

d\nB 

0/21  B 

®22  B 

nB 

Om  1  kd 

Orn'lP 

O' mnB 

(i.i) 


Its  usefulness  in  the  analysis  of  stochastic  models  derives  from  the  ease  with  which 
one  may  describe  the  state  space  of  certain  juxtaposed  stochastic  processes  (c.f.  The¬ 
orem  5.3.5  of  [82]).  Finally,  we  introduce  the  Kronecker  sum  ©  for  square  matrices, 
which  is  defined  as 

A®B  =  A®Ib  +  B®Ia, 
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where  I  a  and  Ib  are  the  identity  matrices  of  the  same  dimensions  as  that  of  A  and 
B ,  respectively. 

Some  important  properties  of  the  Kronecker  product  bear  mentioning  since 
they  are  crucial  to  the  integration  of  expressions  containing  matrices.  We  begin  with 
the  identity 

(A  <g>  C)(B  ®D)  =  AB®  CD ,  (1.2) 

where  all  terms  are  rectangular  matrices  for  whom  the  products  AB  and  CD  are 
defined.  As  in  [82:  p  245],  we  observe  that 

exp  (A)  <g)  exp  (5)  =  exp(A  <g>  /  +  /  0  B)  =  exp(A  ©  B),  (1.3) 

which  is  analogous  to  the  behavior  of  products  of  scalar  exponential  terms. 

A.  1.2  Matrix  Convolution  Products 

One  other  matrix  operation  bears  mentioning  since  it  will  appear  in  our  dis¬ 
cussion  of  M/G/  1-type  processes  in  Chapter  4.  The  matrix  convolution  product  is, 
as  its  name  suggests,  related  to  the  notion  of  the  convolution  of  two  scalar  mass- 
functions.  First,  the  matrix  convolution  (-,-,*)  of  two  matrices  of  distribution  or 
mass-functions,  which  we  shall  denote  by  F(x)  =  [fijix)]  and  G(x)  =  [gij(x)\,  is 
defined  as  the  m  x  m  matrix 

m  m 

[F(x)  *  G(x)]ij  =  /  fik(x  -  Xi)gfcj(xi)  dx i  =  ^  fik(x)  *  gkj(x).  (1.4) 

k=  1  k= 1 

From  this  definition,  we  obtain  the  Kronecker  convolution  product  (•,  *,  ©),  which  is 
explicitly  given  by  the  m2  x  m2  matrix 

F(x)®G(x)  =  [fij(x)  *  G(x)]ij.  (1.5) 
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An  overview  of  the  operation,  related  properties,  and  more  advanced  mathematical 
applications  may  be  found  in  [53]. 

A. 2  PH-Random  Variables 

Phase-type,  or  Pif-distributed  random  variables  define  the  time-to-absorption 
of  a  Markov  process  with  initial  probability  vector  (0O>  4>)  and  with  hnitely-many 
transient  states,  or  phases ,  1, .  ..,m,  and  absorbing  state  m  +  1.  Processes  that 
behave  accordingly  are  absorbed  with  probability  one.  Important  applications  re¬ 
side  in  such  fields  as  reliability  theory,  where  the  absorbing  state  frequently  rep¬ 
resents  equipment  failure,  after  which  the  process  ends  or  it  may  be  restarted  by 
a  repair  or  replacement.  The  restarting  of  the  process  may  be  considered  to  be 
regenerative  (or  at  least  Markov  regenerative  in  the  case  that  the  process  restarts 
with  different  parameters).  This  trait  enables  PH-distributions  to  aptly  describe  the 
Markov-modulated  Poisson  process,  which  features  highly  in  the  queueing  model  of 
this  dissertation. 

The  idea  of  a  PH- distribution  is  an  extension  of  the  earlier  method  of  stages, 
which  was  first  studied  by  Erlang  using  the  distribution  that  bears  his  name.  In  fact, 
the  family  of  Erlang  and  hyperexponential  distributions  are  themselves  special  mani¬ 
festations  of  PH-distributions,  which  in  turn  are  dense  in  the  space  of  all  nondefective 
distributions.  This  fact  is  useful  to  the  approximation  of  arbitrary  (nondefective) 
distributions.  Moreover,  the  PH-distribution  function  possesses  attractive  character¬ 
istics  that  often  culminate  in  computationally  tractable  mathematical  expressions. 
For  these  reasons,  PH-distributions  and  their  matrix- analytic  generalizations  are 
becoming  increasingly  important  in  the  held  of  stochastic  operations  research. 
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A. 2.1  Continuous  PH- Distributions 


To  a  continuous  PH-random  variable  X  we  associate  the  infinitesimal  generator 


Q 


0  0 
t  T 


(1.6) 


of  a  Markovian  process,  where  t  is  a  column  vector  and  T  is  a  square  matrix  such  that 
Te+t  =  0,  both  of  dimension  m,  to  the  process  with  non-absorbing  states  {1, . . . ,  m} 
and  absorbing  state  0.  We  shall  assume  for  the  remainder  of  this  discussion  that  the 
0o  component  of  the  initial  probability  vector  (0Ol  0)  is  equal  to  zero,  since  this  is 
equivalent  to  stating  that  the  corresponding  PH-process  does  not  begin  in  state  0. 

Define  the  matrix  exponential  exp  (A)  to  be  a  function  whose  domain  consists 
of  square  matrices  A  such  that 


exp  (A)  =  ^ 
k= o 


Ak 

fcT 


We  may  now  define  the  c.cl.f.  of  the  PH-distribution  associated  to  the  generator  Q 
as 

F(x)  =  P  {X  <  x}  =  1  —  0>exp  (Tx)e, 


which  we  may,  using  the  fact  that  t  =  —  Te,  differentiate  in  order  to  obtain  the 
density  function  of  X 

f(x)  =  0exp  (Tx)t. 


It  is  clear  from  these  definitions  that  the  PH-random  variable  is  completely  de¬ 
termined  by  the  stochastic  vector  <p  and  the  matrix  T.  Hence,  we  call  [(f),  T]  the 
representation  of  the  PH-distribution  that  we  now  denote  by  PH(<f>,T).  As  noted 
by  Neuts  [83],  Latouche  and  Ramaswami  [66],  and  others,  this  representation  is  by 
no  means  unique,  but  one  can  always  find  (for  a  nondefective  PH-distribution)  a 
representation  such  that  the  probability  of  absorption  from  any  phase  i  =  1 , ,  m 
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is  equal  to  one.  Such  a  representation  is  necessary  and  sufficient  for  the  matrix  T  to 
be  invertible  (see  [66]),  and  hence,  we  may  always  find  a  representation  [(f),  T]  for  a 
nondefective  PH-distribution  for  which  T  is  invertible. 

A. 2.2  Discrete  PH- Distributions 

Discrete  PH-distributions  represent  the  distribution  of  the  time-to-absorption 
of  a  discrete-time  Markov  chain  (DTMC).  They  may  also  be  represented  by  the  pair 
[t,  T ],  except  that  the  matrix  portion  T  of  the  representation  contains  the  probability 
of  transitions  between  nonabsorbing  states  of  an  irreducible  DTMC.  In  other  words, 
the  transition  probability  matrix  of  the  DTMC  is  given  by 

0  0 

P  =  ,  (1.7) 

t  T 

where  t  +  Te  =  e.  Suppose  that  X  ~  PH^fr,  T ).  Since  we  have 

0  0 

e  -  Tke  Tk 

the  mass  function  and  distribution  function  of  X  are  given  by 
P  {X  —  0}  —  To 

P  {x  =  k}  =  TTk~H,  k  >  1 

P  {X  <  k}  =  1  -  rTke ,  k  >  0, 

(see  [66:  Thm  2.5.3]).  Most  importantly,  nondefective  discrete  PH-distributions  are 
also  guaranteed  the  existence  of  a  representation  such  the  I  —  T  is  nonsingular;  in 
other  words,  the  probability  of  absorption  from  any  state  is  equal  to  1. 
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A. 2.3  Properties  of  the  PH- Distribution 

Since  there  will  be  occasion  to  use  PH-distributions  in  conditioning  arguments, 
we  will  give  some  additional  properties. 

Lemma  A.l.  Let  [<p,T]  be  the  representation  of  a  nondefective  continuous  PH- 
distribution.  Then  the  following  holds: 

exp  (Tx)dx  =  (— T)~l .  (1.8) 

Proof.  By  definition, 


F(x )  =  /  f{u)  du 


=  (p  I  exp (Tu)du(-Te). 


(1.9) 


Taking  the  limit  of  (1.9)  as  x  — »  oo  gives  us  the  following  relationship: 


1  =  (p  I  exp (Tu)du(-Te). 


(1.10) 


Since  the  distribution  is  assumed  to  be  nondefective,  we  may  assume  that  T  is 
nonsingular.  We  thus  obtain  our  result  by  substituting  (pie  for  1  in  (1.10)  and 
solving.  □ 

The  convolution  of  two  independent  PH-random  variables  X  and  Y  will  also 
be  required.  The  following  appears  as  Theorem  2.6.1  in  [66]: 

Theorem  A.l.  Let  X  ~  PH{r.T )  arid  Y  ~  PH(/3,B)  be  independent  with  m  and 
n  phases,  respectively.  Their  sum  X  +  Y  ~  PH( 7,  C)  with  m  +  n  phases,  where 


A  =  [r,  r0/3\ 


and 


C 


T  t/3 
0  5 
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A. 3  Markov- Modulated  Poisson  Processes  (MMPP) 


The  MMPPs  are  members  of  a  class  of  processes  termed  doubly- stochastic,  the 
name  taking  its  cue  from  the  fact  that  the  parameters  of  the  statistical  distribution 
of  the  random  variables  of  the  process  are  themselves  random  variables.  MMPPs 
are  considered  a  special  case  due  to  the  fact  that  the  exponential  distribution  of  its 
interarrivals  has  a  parameter  that  itself  randomly  varies  according  to  a  finite-state 
Markov  chain.  In  other  words,  if  the  modulating  Markov  process  {Z(t)  :  t  >  0}  has 
the  state  space  S  =  {1,2,. . . , m},  whenever  Z(t)  =  j,  arrivals  occur  acording  to  a 
Poisson  process  with  rate  A  j,  j  G  S.  An  excellent  and  highly-useful  overview  of  the 
MMPP  and  its  properties  may  be  found  in  [40]. 

A. 3.1  The  Markov- Modulated  Poisson  Process  as  a  Markov  Renewal  Process 

It  is  a  well-established  fact  that  the  standard  Poisson  process  {N(t)  :  t  >  0}  is  a 
renewal  process  characterized  by  the  i.i.d.  nature  of  the  interarrival  times.  However, 
as  in  [40,  82],  an  MMPP  with  exponential  rates  contained  in  the  vector  rj  is  not  a 
standard  renewal  process  due  to  the  dependence  of  the  interarrival  durations  upon 
the  evolution  of  the  modulating  process.  Nevertheless,  it  is  still  possible  to  describe 
the  MMPP  instead  as  a  Markov  renewal  process  or  semi-Markov  process  for  which  we 
require  only  that  the  increments  of  the  process  behave  in  accordance  with  the  Markov 
property.  In  order  to  see  this,  consider  the  MMPP  at  arrival  epochs  tn  >  0  and  let 
the  state  of  the  modulating  process  at  tn  be  Zn  =  Z(t+)  and  rn  =  tn  —  tn-i.  Then  the 
process  {( Zn,rn )  :  n  >  0}  defines  a  Markov  renewal  sequence.  In  other  words,  since 
we  embed  the  sequence  at  transitions  of  a  Markov  process,  the  Markov  property  may 
be  said  to  hold  for  the  increments  of  the  embedded  sequence.  Most  importantly,  it  is 
a  well-known  property  of  Markov  renewal  sequences  that  {Zn  :  n  >  0}  itself  defines 
a  discrete-time  Markov  chain. 
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We  next  introduce  the  kernel  K(x )  of  the  semi-Markov  process,  which  is  defined 
to  be  the  matrix  with  entries  given  by 

Kij(x)  =  P  {Zn  =  j,rn  <x  \  Zn_i  =  i}  ,  x  >  0,  n  >  1, 

where  i  and  j  belong  to  the  state  space  of  the  Markov  chain  {Zn  :  n  >  0}.  The 
importance  of  the  kernel  lies  in  the  fact  that  P  =  K( oo)  is  the  transition  probability 
matrix  of  the  MMPP  embedded  at  arrival  times.  Thus,  in  utilizing  Markov  renewal 
theory,  we  will  have  enabled  the  use  of  elementary  methods  defined  for  Markov  chains 
in  the  analysis  of  the  steady-state  conditions  of  a  non-Markovian  process.  We  will 
use  this  property  later  in  constructing  an  embedded  Markov  process  that  will  serve 
as  a  proxy  for  the  analysis  of  a  non-Markovian  M/G /1-type  queueing  system. 

A. 3.2  The  Relationship  of  the  PH- Distributions  to  the  Markov-Modulated,  Poisson 
Process 

The  utility  of  the  PH-distribution  for  our  queueing  model  rests  in  its  association 
to  the  MMPP.  As  mentioned  previously,  inter-event  times  in  a  MMPP  may  no  longer 
be  characterized  by  a  single  exponential  distribution.  Neuts  [83]  has  shown  that 
Markovian  processes  whose  rate  parameters  rj  vary  lexicographically  according  to 
the  states  of  an  external  Markov  chain  with  generator  Q  possess  inter-event  times 
distributed  as  PH(<f>),Tn,  where 


Tv  =  Q  -  A  (77) . 

Thus,  the  distribution  of  time  between  the  arrival  of  primary  customers  to  the  retrial 
queue  is  PH((f)x,  T\)  for  a  suitable  choice  of  initial  probability  vector  <px.  In  this 
light,  it  is  clear  that  the  MMPP  is  a  PH-renewal  process ,  which  belongs  to  a  class  of 
point  processes  termed  versatile  by  Neuts  and,  more  recently,  the  Markovian  arrival 
process  (MAP)  or  the  batch  Markovian  arrival  process  (BMAP). 
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Armed  with  this  knowledge,  it  is  now  possible  to  fully  describe  the  counting 
process  associated  to  the  MMPP.  Define  N(t)  to  be  the  number  of  renewals  (i.e. 
arrivals)  in  (0,  t],  and  set 

=  P{N{t)  =  v,Z(t)=f\N(0)  =  0,Z(0)=j},  (1.11) 

where  j  and  f  are  in  the  state  space  of  the  modulating  Markov  chain  {Z(t)  :  t  >  0} 
and  v  e  Z+.  As  in  [40],  (1.11)  obeys  the  forward  Chapman-Kolmogorov  equations 

^'(0  ,t)  =  ^(0  ,t)Tv,  (1.12) 

^'(M)  =  k>  1,  (1.13) 

which,  of  course,  directly  implies  that 

&v(0,t)  =  exp  (Tvx). 

Equations  (1.12)  and  (1.13)  may  also  be  used  to  derive  the  probability  generating 
function  $?7*,  which  is  stated  in  terms  of  the  following  lemma: 

Lemma  A. 2.  Consider  the  MMPP  with  the  exponential  arrival  rate  parameters 
contained  in  the  vector  77  and  infinitesimal  generator  Q  of  the  random  environment. 
Ift  >  0  is  a  real  number,  then  the  probability  generating  function  $ v*(z,t )  of$v(v,t) 
is  given  by 

®v*(z,t)  =  exp[Q  —  (1  —  z)A(rj)],  (1.14) 

where  z  is  a  number  that  resides  in  the  interval  (0, 1). 

We  shall  require  Lemma  A. 2  in  constructing  the  entries  of  the  semi-Markov  kernel 
Q*(x)  for  the  queueing  system  that  we  shall  analyze  in  Section  5.2. 

For  completeness,  we  consider  the  value  of  the  initial  probability  vector  of  a 
generic  MMPP  with  rate  vector  77.  The  choice  of  this  vector  determines  the  start 
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time  of  the  MMPP,  as  well  as  the  state  in  which  the  process  begins.  There  is  no 
restriction  on  the  choice  of  the  initial  probability  vector,  save  that  its  elements  must 
sum  to  unity.  However,  there  are  two  commonly  used  methods  in  determining  its 
value.  The  first  is  the  selection  that  results  in  the  environment-stationary  version  of 
the  MMPP.  This  corresponds  to  the  selection  of  the  steady-state  probability  vector 
p  of  the  random  environment,  which  subsequently  starts  the  MMPP  at  some  point 
in  time  at  which  the  environment  achieves  equilibrium.  The  other  version  of  the 
MMPP  is  known  as  the  time- stationary  version.  Here,  the  initial  probability  vector 
is  chosen  in  such  a  way  that  the  Markov  renewal  sequence  corresponding  to  the 
MMPP  is  itself  in  equilibrium.  As  it  turns  out,  both  versions  are  stochastically 
equivalent  (see  [82:  Thm  5.3.3]). 

In  characterizing  the  semi-Markov  kernel  Q*  (. x )  for  queueing  systems  with  more 
than  one  MMPP  (or  PH)  arrival  stream,  it  becomes  necessary  to  condition  upon 
the  order  of  arrivals  in  a  regenerative  cycle.  For  this  purpose,  it  is  necessary  to 
determine  riy=i  P{Xi  <  Xj,  i  e  {1, . . . ,  N},  for  a  finite  set  of  PH-random  variables 
with  representations  [4>t,  Tj\.  Suppose  that  N  =  2  and  that  Ft(x )  =  1  —  4>iexp(Tx)e 
are  the  c.d.f.s  corresponding  to  the  random  variables  X±, . . . ,  X]y.  We  compute  these 
probabilities  by  conditioning  upon  the  values  of  F2(x),  noting  that  the  dimension  of 
e,  the  column  vector  of  ones,  varies  accordingly,  even  though  this  is  not  explicitly 
indicated. 


P{X  !<X2}  =  /  F1(x)dF2(x) 


(1  —  <fi1exp(Tix)e)  (— <fi2  exp(T2x)T2e)  dx 


=  1  +  (01  ®  <£2) 

=  1  +  (0i  ®  02) 


exp  (Tire)  ®  exp(T2a;)  dx 


Uo 


L./o 


(e  <g)  T2e) 


exp[(Ti  ©  T2)x\  dx 


(e  ©  T2e) 
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1  +  ( 4>i  ©  (f> 2 )  [—  (Ti  ©  T2)\  1  (e  ©  T2e). 


(1.15) 


The  equality 

4>x  exp{Tix)(p2  exp(T2x)T2e  dx 

poo 

=  {(f)l  ©  <p2)  /  exp(Tj£)  ©  exp(T2a;)  dx  (e  ©  T2e) 

Uo  J 

follows  from  first  noticing  that  the  ordinary  product  of  two  scalars  is  also  a  tensor 
product,  to  which  we  may  then  apply  the  product  rule  (1.2).  In  other  words, 

< pi  exp(Tix)ec/)2  exp {T2x)T2e 

=  (01exp(Tia;)e)  ©  (</>2  exp(T2x)T2e) 

=  (01  ©  02)(exp(Tia;)  ©  exp(T2a;))(e  ©  T2e ) 

=  (01  ©  02)  exp[(Ti  ©  T2)x](e  ©  T2e),  (1-16) 

with  the  final  equality  resulting  from  the  identity  (1.3). 

In  considering  the  MMPP  in  the  context  of  the  MMPP/G/ 1  queue,  we  obtain 
a  Markov  renewal  process  by  considering  the  number  of  arrivals  to  the  queue  at 
instants  Sn  =  T+  just  after  the  nth  departure.  Let  Yn  be  the  size  of  the  system  at 
Tn  and  let  rn  =  Sn  —  Sn_ i.  In  this  way,  we  obtain  the  Markov  renewal  sequence 
{(K„,  Zn,  rn)  :  n  >  0},  which  defines  the  kernel  matrix  Q*(x )  consisting  of  the  block 
elements 

Ajj'{x)  =  [Qii'{x)\j:jl  =  P{Yn  =  i\  Zn  =  j’,  rn  <  x  |  Yn_ i  =  i,  Zn_t  =  j} 
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if  i  >  0  and 


Bjj>(x )  =  [Q*0i'(x)]j:j,  =  P  { Yn  =  i.%  Zn  =  j\  rn  <  x  |  Yn_!  =  0,  Zn_x  =  j}  , 

where  j,jf,  and  x  are  defined  as  before  and  i  and  i'  are  nonnegative  integers  that 
correspond  to  the  number  in  the  system.  The  matrix  P  =  B{ oo)  is  the  transition 
probability  matrix  of  the  Markov  chain  embedded  just  after  departure  instants.  The 
kernel  Q*(x)  for  M/G/  1-type  processes  assumes  a  distinctive  structure  which,  like 
the  tridiagonal  form  of  the  QBD  generator,  a  set  of  class-based  methods  that  may 
be  applied  to  a  wide  spectrum  of  related  models. 
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